How to import from a data file a numpy structured array

Question

i'm trying to create an array which has 5 columns imported from a data file. The 4 of them are floats and the last one string.

The data file looks like this:

5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
4.7,3.2,1.3,0.2,Iris-setosa
4.6,3.1,1.5,0.2,Iris-setosa
5.0,3.6,1.4,0.2,Iris-setosa
5.4,3.9,1.7,0.4,Iris-setosa
4.6,3.4,1.4,0.3,Iris-setosa
5.0,3.4,1.5,0.2,Iris-setosa

I tried these:

data = np.genfromtxt(filename, dtype = "float,float,float,float,str", delimiter = ",")

data = np.loadtxt(filename, dtype = "float,float,float,float,str", delimiter = ",")

,but both codes import only the first column.

Why? How can i fix this?

Ty for your time! :)

Take a look at the `usecols` keyword in either of the functions you tried. — RuthC, Dec 02 '18 at 15:57
I tried! the same... The default that now i am using it supposed to take all the columns — crystal, Dec 02 '18 at 15:59
@stevi, don't post input data as an image, post a testable text fragment — RomanPerekhrest, Dec 02 '18 at 16:02
Your `dtype` should work, except for the size of the final string field. — hpaulj, Dec 02 '18 at 17:09
Did you actually look at `data` and its `dtype`? Or just see the `shape` (which will be 1d)? — hpaulj, Dec 02 '18 at 17:52

B. M. · Accepted Answer · 2018-12-02T16:43:18.310

1

You must specify correctly the str type : "U20" for exemple for 20 characters max :

data = np.loadtxt('data.txt', dtype = "float,"*4 + "U20", delimiter = ",")

seems to work :

array([( 5.1,  3.5,  1.4,  0.2, 'Iris-setosa'),
       ( 4.9,  3. ,  1.4,  0.2, 'Iris-setosa'),
       ( 4.7,  3.2,  1.3,  0.2, 'Iris-setosa'),
       ( 4.6,  3.1,  1.5,  0.2, 'Iris-setosa'),
       ( 5. ,  3.6,  1.4,  0.2, 'Iris-setosa'),
       ( 5.4,  3.9,  1.7,  0.4, 'Iris-setosa'),
       ( 4.6,  3.4,  1.4,  0.3, 'Iris-setosa'),
       ( 5. ,  3.4,  1.5,  0.2, 'Iris-setosa')],
      dtype=[('f0', '<f8'), ('f1', '<f8'), ('f2', '<f8'), ('f3', '<f8'), ('f4', '<U20')])

An other method using pandas give you an object array, but this slow down further computations :

In [336]: pd.read_csv('data.txt',header=None).values
Out[336]: 
array([[5.1, 3.5, 1.4, 0.2, 'Iris-setosa'],
       [4.9, 3.0, 1.4, 0.2, 'Iris-setosa'],
       [4.7, 3.2, 1.3, 0.2, 'Iris-setosa'],
       [4.6, 3.1, 1.5, 0.2, 'Iris-setosa'],
       [5.0, 3.6, 1.4, 0.2, 'Iris-setosa'],
       [5.4, 3.9, 1.7, 0.4, 'Iris-setosa'],
       [4.6, 3.4, 1.4, 0.3, 'Iris-setosa'],
       [5.0, 3.4, 1.5, 0.2, 'Iris-setosa']], dtype=object)

edited Dec 02 '18 at 16:43

answered Dec 02 '18 at 16:27

B. M.

18,243
2
35
54

Seems correct but try table.shape, its shape is 8 while i want (8,5) :/ It looks like an array with one column that has tuples in each row. – crystal Dec 02 '18 at 16:38
It's normal with structured array. Pandas can help you for what you want. edited. – B. M. Dec 02 '18 at 16:40
`dtype=None` should deduce the field dtypes correctly. – hpaulj Dec 02 '18 at 16:51
@hpaulj I obtain with loadtxt : ValueError: could not convert string to float: b'Iris-setosa' – B. M. Dec 02 '18 at 16:58
`genfromtxt` does the auto dtype better than `loadtxt`. – hpaulj Dec 02 '18 at 17:08

How to import from a data file a numpy structured array

1 Answers1