1

I am using astropy to manipulate a FITS table, and I would like to remove all the rows that contain a nan.

With a fits table stored in mytable, I tried the following

data = np.lib.recfunctions.structured_to_unstructured(np.array(mytable))
idx = []
for i, line in enumerate(data):
    for e in line:
        if e !=e:
            idx.append(i)
data = Table([data[i] for i in range(len(data)) if i not in idx])

which seems to work, but is rather clunky. Is there a more pythonic way to do this?


I want to remove the entire line if one of the elements in that line is a nan. This question is different, since it is about removing single elements.

usernumber
  • 1,958
  • 1
  • 21
  • 58

1 Answers1

2

A more num-pythonic way to do this would be:

data = np.lib.recfunctions.structured_to_unstructured(np.array(mytable))
has_nan = np.any(np.isnan(data), axis=1)
mytable_no_nan = mytable[~has_nan]

For a more general and astro-pythonic way (assuming mytable is an astropy Table):

has_nan = np.zeros(len(mytable), dtype=bool)
for col in mytable.itercols():
    if col.info.dtype.kind == 'f':
        has_nan |= np.isnan(col)
mytable_no_nan = mytable[~has_nan]
Tom Aldcroft
  • 2,339
  • 1
  • 14
  • 16