Adding datetime field to recarray — revisited

Question

I have a recarray obtained using np.recfromcsv, one of its fields being an array of strings

In [220]: data['data']
Out[220]:
array(['2019-12-31', '2020-01-01', '2020-01-02', ..., '2020-03-01',
       '2020-03-02', '2020-03-10'], dtype='<U10')

I want to plot some of the other fields against the dates and, as far as I know, I have to convert the date strings to datetime objects

In [221]: dt = np.array([dateutil.parser.parse(d) for d in data['date']])
     ...: dt
Out[221]:
array([datetime.datetime(2019, 12, 31, 0, 0),
       datetime.datetime(2020, 1, 1, 0, 0),
       datetime.datetime(2020, 1, 2, 0, 0), ...,
       datetime.datetime(2020, 3, 1, 0, 0),
       datetime.datetime(2020, 3, 2, 0, 0),
       datetime.datetime(2020, 3, 10, 0, 0)], dtype=object)

Because it's handy to filter on some field value before plotting

In [222]: filtered = data[data['comune']=='Busto Arsizio']

I'd like to add the dt array to data as a new field, but…

In [223]: np.lib.recfunctions.append_fields(data, 'dt', dt)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-224-b048aaed3e78> in <module>
----> 1 np.lib.recfunctions.append_fields(data, 'dt', dt)

<__array_function__ internals> in append_fields(*args, **kwargs)

~/lib/miniconda3/lib/python3.7/site-packages/numpy/lib/recfunctions.py in append_fields(base, names, data, dtypes, fill_value, usemask, asrecarray)
    716     if dtypes is None:
    717         data = [np.array(a, copy=False, subok=True) for a in data]
--> 718         data = [a.view([(name, a.dtype)]) for (name, a) in zip(names, data)]
    719     else:
    720         if not isinstance(dtypes, (tuple, list)):

~/lib/miniconda3/lib/python3.7/site-packages/numpy/lib/recfunctions.py in <listcomp>(.0)
    716     if dtypes is None:
    717         data = [np.array(a, copy=False, subok=True) for a in data]
--> 718         data = [a.view([(name, a.dtype)]) for (name, a) in zip(names, data)]
    719     else:
    720         if not isinstance(dtypes, (tuple, list)):

~/lib/miniconda3/lib/python3.7/site-packages/numpy/core/_internal.py in _view_is_safe(oldtype, newtype)
    459
    460     if newtype.hasobject or oldtype.hasobject:
--> 461         raise TypeError("Cannot change data-type for object array.")
    462     return
    463

TypeError: Cannot change data-type for object array.

What I'm missing?

I know that this question has the same title but it deals, afaict, with a different backtrace/problem.

Clearly the `dt` is object dtype array. For one reason or other this `append_fields` function can handle that. `recfunctions` is not an actively developed or robust subpackage (except for some recent additions to handle multiple-field indexing). The do-it-yourself append in my old answer is still your best alternative. — hpaulj, Jul 01 '20 at 16:47
@hpaulj OK for the do-it-yourself append in your old answer. I'll comment further under it. — gboffi, Jul 02 '20 at 06:14

Adding datetime field to recarray — revisited

0 Answers0