Questions tagged [netcdf4]

Questions related to NetCDF (Network Common Data Form) format and corresponding libraries/implementations may have this tag.

In 2008, the netCDF4 format was added to support per-variable compression, multiple unlimited dimensions, more complex data types, and better performance, by layering an enhanced netCDF access interface on top of the HDF5 format.

At the same time, a fourth format variant, netCDF-4 classic model format, was added for users who needed the performance benefits of the new format (such as compression) without the complexity of a new programming interface or enhanced data mode.

Every netCDF-4 or netCDF-4 classic model file can be read or written by the HDF5 library version 1.8 or later, so in that respect netCDF-4 files are already HDF5 files and need no conversion.

According to netCDF FAQ installing the netCDF4 is required for any of the following situations:

  • you need to access netCDF data that makes use of netCDF-4 compression or chunking
  • you need to access data in all netCDF formats including netCDF-4 or netCDF-4 classic model formats
  • you need to write non-record variables larger than 4GiB or record variables with more than 4GiB per record
  • you are installing netCDF to support other software packages that require netCDF-4 features
  • you want to write data that takes advantage of compression, chunking, or other netCDF-4 features
  • you want to be able to read netCDF-4 classic model data with no changes to your current software except relinking with the new library
  • you want to benchmark your current applications with the new libraries to determine whether the benefits are significant enough to justify the upgrade
  • you need to use parallel I/O with netCDF-4 or netCDF-4 classic files
651 questions
5
votes
1 answer

Python line_profiler not finding module

I recognize that this is an installation failure on my part, and I'm sorry to lay this uninteresting and inconsequential question at your feet, but for the life of me I can't manage to figure out what is going wrong and I've run out of ideas. I'm…
4
votes
3 answers

Python Xarray ValueError: unrecognized chunk manager dask - must be one of: []

I am using xarray for combining multiple netcdf files using xarray.open_mfdataset. But I get the error while running the command, below are the commands and error. nc_all = xarray.open_mfdataset(files,combine = 'nested', concat_dim="time") files =…
lsr729
  • 752
  • 2
  • 11
  • 25
4
votes
1 answer

How to merge/ combine time variable in multiple netcdf

I have a lot of Sea Surface Temperature NetCDF files with the same lat and lon dimensions, but different time variables. I want to try to combine it into 1 NetCDF file by combining the time variables because the time variables in each netcdf file…
ExHunter
  • 307
  • 4
  • 11
4
votes
0 answers

How to bin a netcdf data using xarray

I have some spatiotemporal data derived from the CHIRPS Database. It is a NetCDF that contains daily precipitation for all over the world with a spatial resolution of 1x1km2. The DataSet possesses 3 dimensions ('time', 'longitude', 'latitude'). I…
Philipe Riskalla Leal
  • 954
  • 1
  • 10
  • 28
4
votes
0 answers

Difference in sys.getsizeof, nbytes, and DataArray.size in xarray

I'm curious about the differences of some methods of checking how big an xarray DataSet is as I decide whether or not I can successfully load it all into memory. I open up a set of precipitation netcdfs with: imerg_xds =…
clifgray
  • 4,313
  • 11
  • 67
  • 116
4
votes
2 answers

How to know if a sliced xarray Dataset/DataArray is empty?

I am applying slicing and aggregation operations over Netcdf files in Python language. One of the solutions for working with this kind of file is to use the Xarray library. I am still new to the library functionalities, so I would like to know…
Philipe Riskalla Leal
  • 954
  • 1
  • 10
  • 28
4
votes
0 answers

Unknown file format error when trying to read netcdf file in python 3.7

I am trying to read in a netcdf file with Spyder(python 3.7) from the Anacondas package and I am getting the error message: [Errno -51] NetCDF: Unknown file format: b'AMSR2-MBT_v2r0_GW1_s201907081630360_e201907081809340_c201907081825230.nc' I've…
BBUSH
  • 49
  • 3
4
votes
2 answers

R crashes while opening netcdf file

I downloaded a netcdf file and trying to open it in R. Here's my code download.file("https://data.giss.nasa.gov/impacts/agmipcf/agmerra/AgMERRA_1980_prate.nc4",destfile = "AgMERRA_1980_prate.nc4", method="libcurl") I want to open the netcdf file…
89_Simple
  • 3,393
  • 3
  • 39
  • 94
4
votes
1 answer

Downloading NetCDF files with R: Manually works, download.file produces error

I am trying to download a set of NetCDF files from: ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nwm/prod/nwm.20180425/medium_range/ When I manually download the files I have no issues connecting, but when I use download.file and attempt to connect…
nick
  • 51
  • 5
4
votes
1 answer

How to get indices of valid values of numpy masked array

I have a netCDF4 dataset representing multiple matrices of the same dimension (551, 146), one matrix (M1) contains longitude values, another matrix (M2) contains latitude values. Each matrix is a numpy masked array. Given a lon/lat tuple, (A, B), I…
jollyroger
  • 659
  • 1
  • 10
  • 19
4
votes
2 answers

How to save xarray.DataArray with complex128 data to netcdf

I have some complex data (numpy dtype complex128) in an xarray data set which I want to save with to_netcdf. I get the following error: TypeError: illegal primitive data type, must be one of dict_keys(['S1', 'i1', 'u1', 'i2', 'u2', 'i4', 'u4',…
Anna Krause
  • 43
  • 1
  • 4
4
votes
2 answers

add variable to an existing Netcdf4 file using Python

I'd like to add an additional variable containing the Julian dates to an existing set of NetCDF climate data files. I've done a bit of python scripting but it has been a while so I'm rather rusty. After reading the "netCDF4 module" documentation…
MapleMatrix
  • 99
  • 1
  • 2
  • 5
4
votes
3 answers

Monthly average from netCDF files in R

I have one netCDF file (.nc) with 16 years(1998 - 2014) worth of daily precipitation (5844 layers). The 3 dimensions are time (size 5844), latitude (size 19) and longitude (size 20) Is there a straightforward approach in R to compute for each…
Jobbo90
  • 175
  • 1
  • 11
3
votes
0 answers

Issue importing netCF4 with python 3.9

I'm trying to import netCDF4 and got this error message: from netCDF4 import Dataset from netCDF4 import * File "/home/username/.local/lib/python3.9/site-packages/netCDF4/init.py", line 3, in from ._netCDF4 import * ModuleNotFoundError: No…
user496181
  • 141
  • 7
3
votes
2 answers

Averaging multiple netCDF4 files with python

I am a bit of a netCDF in python noob so please excuse this noob question. I have a folder filled with circa 3650 netCDF4 files. One file per day for a decade. the niles are named yyyymmdd.nc (e.g. 20100101,20100102,20100103,etc.). Each .nc file…
kawakawa
  • 75
  • 1
  • 4
1
2
3
43 44