numpy.genfromtxt Error with convert function

Question

I've data set which contains dates,data as shown below. the data is in mixed format and what I want to is to read date in datetime format and data in float and to store in a numpy matrix.

CST,Max Tempe atu eC,Mean Tempe atu eC,Min Tempe atu eC,Dew PointC,MeanDew PointC,Min DewpointC,Max Humidity, Mean Humidity, Min Humidity, Max Sea Level P essu ehPa, Mean Sea Level P essu ehPa, Min Sea Level P essu ehPa, Max Visi ilityKm, Mean Visi ilityKm, Min Visi ilitykM, Max Wind SpeedKm h, Mean Wind SpeedKm h, Max Gust SpeedKm h,P ecipitationmm, CloudCove , Events,WindDi Deg ees
2014-12-1,33,28,22,24,23,21,94,81,53,1017,1012,1009,19,12,10,19,6,,0.00,1,,23
2014-12-2,34,28,22,25,22,21,96,74,43,1015,1011,1007,19,12,10,23,10,,0.00,0,,300
2014-12-3,34,28,21,23,21,15,89,71,33,1013,1010,1008,19,13,10,11,5,,0.00,0,,314

I've tried numpy.genfromtxt as show below

def c_date(dstr):
    return dt.datetime.strptime(dstr, '%Y-%m-%-d')

A=np.genfromtxt('test.csv',names=('CST',)+ tuple('col{i}'.format(i=i) for i in range(1,24)),converters={'CST': c_date}, dtype=None)

but it gives below shown error.

ConverterError: Converter #0 is locked and cannot be upgraded: (occurred line #1 for value 'CST,Max')

Why does it throw such an Error? Could anyone suggest a solution ? Any thoughts and all suggestions will be highly appreciated.

Aside: if you're working with real data and named columns, you're probably going to have an easier go using [`pandas`](http://pandas.pydata.org) rather than bare `numpy`. — DSM, Dec 23 '14 at 07:11

hpaulj · Accepted Answer · 2014-12-23T07:07:48.803

First there's an extra dash in the c_date function. Correcting that, I get the following to work:

A = np.genfromtxt(txt,names=True,converters={'CST': c_date}, dtype=None,delimiter=',')

You are missing the delimiter parameter.

This also works:

A = np.genfromtxt(txt,names=('CST',)+ tuple('col{i}'.format(i=i) for i in range(1,24)),
    converters={'CST': c_date}, dtype=None,delimiter=',',skip_header=1)

Here I added skip_header. When you give your own names, you need to skip the names in the header line. The obscure error results, apparently, from trying to read the header line as though it were data.

A=np.genfromtxt(txt,converters={0: c_date}, dtype=None,delimiter=',',skip_header=1)

generates automatic names, similar to yours, except ['f0','f1',etc]

numpy.genfromtxt Error with convert function

1 Answers1