1

I read netCDF files through netCDF4 into python. I get arrays with invalid values that print as --, for example:

[[[-- -- -- ..., 0.0 0.0 0.0]
  [-- -- -- ..., 0.0 0.0 0.0]
  [-- -- -- ..., 0.0 0.0 0.0]]

 [[-- -- -- ..., 0.0 0.0 0.0]
  [-- -- -- ..., 0.0 0.0 0.0]
  [-- -- -- ..., 0.0 0.0 0.0]]]

Where do these values originate from, and how could I identify (and replace) them? Things such as numpy.isnan would return me -- as result, too. I can use these arrays with -- later without crash (I can for example plot them), but they seem to be problematic for some operations.

For example, I used to do something like 

numpy.mean(myarray, axis=(1,2)) # the tuple for `axis` is not very regular

on myarray originating from my netCDF files before without any problem, but I get TypeError: tuple indices must be integers, not tuple and a crash with these weird ---filled arrays...

I am using python 2.7.9 (and, in case relevant for the printing to screen format, PyCharm for code editing).

ztl
  • 2,512
  • 1
  • 26
  • 40

1 Answers1

3

What you have here is a masked array, where -- represent the masked elements.

Just usemyarray.data and myarray.mask to get the data and the invalid elements, respectively.

Most likely, a masked numpy array was intentionally saved in that netcdf4 file, probably for a reason. So I would simply keep using this masked array as it is. As you noticed, most Numpy/Scipy/Matplotlib functions (e.g. numpy.mean) have a specific logic for handling them.

rth
  • 10,680
  • 7
  • 53
  • 77
  • Thanks @rth, of course this is indeed a masked array (I was puzzled by the netCDF origin). Thanks - yet I am still getting the error with `numpy.mean` with tuple as axis argument, even though I updated from `1.9.1` to `1.9.2` like you. Still investigating this... ??? – ztl May 13 '15 at 08:01
  • For the `numpy.mean` trouble, I opened a new question as it remains enigmatic to me: http://stackoverflow.com/questions/30209624/numpy-mean-used-with-a-tuple-as-axis-argument-works-for-one-typeerror-for – ztl May 13 '15 at 08:33
  • @ztl No, sorry, I might have make a mistake while running the test, I do get the exception as well when using a tuple for the axis in masked arrays. – rth May 13 '15 at 10:19