Numpy data conversion after txt reading with genfromtxt

Question

Let me say I created a file with this three lines

A\tB\tC
name1\t1\t2
name2\t1.1\t2.2

where \t corresponds to the delimiter. I read it using this numpy function

data = np.genfromtxt('test.txt', delimiter='\t', dtype=None, encoding='ascii')

Data is a numpy nd array with shape (3,3). I would like to rearrange it into different data structures such as

fileHeader = data[0, :] 
names = data[1:, 0]
values = data[1:, 1:]

fileHeader and names should be list of strings or np.str_ without the character ' leading and trailing. values should be a nd array of float64 without the character ' leading and trailing.

How can I make this conversion? Thank you all in advance!

hpaulj · Answer 1 · 2022-03-17T21:36:08.667

Your code, showing the results (which you should have done!):

In [1]: txt = """A\tB\tC\t
   ...: name1\t1\t2\t
   ...: name2\t1.1\t2.2\t""".splitlines()

In [4]: data = np.genfromtxt(txt, delimiter="\t", dtype=None, encoding="ascii")
In [5]: data
Out[5]: 
array([['A', 'B', 'C', 'False'],
       ['name1', '1', '2', 'False'],
       ['name2', '1.1', '2.2', 'False']], dtype='<U5')
In [6]: fileHeader = data[0, :]
   ...: names = data[1:, 1]
   ...: values = data[1:, 1:]
In [7]: fileHeader
Out[7]: array(['A', 'B', 'C', 'False'], dtype='<U5')
In [8]: names
Out[8]: array(['1', '1.1'], dtype='<U5')
In [9]: values
Out[9]: 
array([['1', '2', 'False'],
       ['1.1', '2.2', 'False']], dtype='<U5')

So what's the problem?

'A' is the normal display of a string. The False' is filler for the trailing empty field (after the last \t).

We could stript off the False with:

In [21]: data = data[:, :-1]
In [22]: data
Out[22]: 
array([['A', 'B', 'C'],
       ['name1', '1', '2'],
       ['name2', '1.1', '2.2']], dtype='<U5')

and convert the numbers to float with:

In [23]: data[1:, 1:]
Out[23]: 
array([['1', '2'],
       ['1.1', '2.2']], dtype='<U5')
In [24]: data[1:, 1:].astype(float)
Out[24]: 
array([[1. , 2. ],
       [1.1, 2.2]])

Thank you for your quick answer. I slightly modified my answer removing the _\_t from each row. The conversion of the _values_ variable into a float numpy array is perfectly achieved with the astype command. However the _fileHeader_ and the _names_ variables remained numpy ndarray. Could you please suggest a way to convert both of them into a list of string without the ' character leading and trailing each element? — eljamba, Mar 17 '22 at 21:58

Numpy data conversion after txt reading with genfromtxt

1 Answers1