1

Is there a graceful way of handling None values in a conversion of a list of tuples to a numpy recarray using the single call to np.rec.fromrecords? Assuming I know what I want the missing value to be (e.g. -1 for integers), how do I catch and handle the below contrived example:

import numpy as np
a = [('Bob', 40, 3.14), ('Sue', 38, 6.28), ('Jim', None, 9.42)]
dtype = [('NAME', 'S10'), ('AGE', np.int32), ('SCORE', np.float64)]
try:
    b = np.rec.fromrecords(a, dtype=dtype)
except TypeError:
    # Convert None to 0 for AGE field here instead of raising the error
    raise TypeError('Caught a TypeError')

I'm guessing that I would have to do this on a per-field basis in order to avoid missing true TypeErrors elsewhere in the recarray. Is there any way of isolating where (ie. what fields) in the recarray I want this conversion to apply. My real use case is converting pyodbc records to numpy recarrays.

Michał Šrajer
  • 30,364
  • 7
  • 62
  • 85
grovduck
  • 406
  • 5
  • 13
  • 1
    Against what data source (Oracle, SQL Server, etc.) are the pyodbc calls made, and can the query be changed? – Bryan Aug 19 '11 at 20:22
  • 1
    Against SQL Server. I think I understand what you're saying, ie. use CAST to convert NULLs to -1 on the DB side. That makes sense. – grovduck Aug 29 '11 at 21:19

1 Answers1

1

Return -1 for the NULL column values using the database query, something like this:

SELECT COALESCE(ColumnName, -1) FROM Schema.Table;

This will return -1 for ColumnName values that are NULL, otherwise the actual value is returned. Documentation for COALESCE here if needed. This allows you to provide a NULL replacement value only for the columns you require, and will not mask TypeError exceptions that you should care about.

Bryan
  • 17,112
  • 7
  • 57
  • 80
  • great, thanks for the help. This seems to be a more reasonable way of handling this for my use case. – grovduck Sep 07 '11 at 22:35