genfromtxt returning -1 and nan rows

Question

I am trying to read a txt file with numpy and I have the following code

import numpy as np
def parsefile(filename):
    return np.genfromtxt(filename,
                     delimiter = dly_delimiter,
                     usecols = dly_usecols,
                     dtype = dly_dtype,
                     skip_header = 4,
                     names = dly_names)
dly_delimiter = [7,5,9,13,10,13,9,13,10,13,10]
dly_usecols = [1,7]
dly_dtype = [np.int32, np.float64]
dly_names = ['node number', 'z_force']
force = parsefile('nodfor')
print(force)

And the following comes out

[(-1, nan) (-1, nan) (-1, nan) ..., (-1, nan) (-1, nan) (-1, nan)]

But what I am interested in are the number after 'nd#' and the number after 'zforce'.

[the format of the txt file is like this] and this is a snippet of the txt file (two lines):

 nd#   39584  xforce=   0.0000E+00   yforce=   0.0000E+00  zforce=   
0.0000E+00   energy=   0.0000E+00   setid =       1
 nd#   39585  xforce=   0.0000E+00   yforce=   0.0000E+00  zforce=   
0.0000E+00   energy=   0.0000E+00   setid =       1

Is this a known issue. How can I fix this?

As @hpaulj notes in his answer, `'#'` is the default comment character. Add the argument `comments=None`, and your code works for me. — Warren Weckesser, Jul 24 '17 at 05:22

hpaulj · Accepted Answer · 2017-07-24T05:22:13.843

My first try misses most of the text because of the comment characters:

In [44]: txt = b""" nd#   39584  xforce=   0.0000E+00   yforce=   0.0000E+00  zforce=   0.0000E+00   energy=   0.0000E+00   setid =       1
    ...:  nd#   39585  xforce=   0.0000E+00   yforce=   0.0000E+00  zforce=   0.0000E+00   energy=   0.0000E+00   setid =       1
    ...: """
In [45]: data=np.genfromtxt(txt.splitlines(),dtype=None)
In [46]: data
Out[46]: 
array([b'nd', b'nd'],
      dtype='|S2')

turn off comments:

In [53]: data=np.genfromtxt(txt.splitlines(),dtype=None, comments=None)
In [54]: data
Out[54]: 
array([ (b'nd#', 39584, b'xforce=',  0., b'yforce=',  0., b'zforce=',  0., b'energy=',  0., b'setid', b'=', 1),
       (b'nd#', 39585, b'xforce=',  0., b'yforce=',  0., b'zforce=',  0., b'energy=',  0., b'setid', b'=', 1)],
      dtype=[('f0', 'S3'), ('f1', '<i4'), ('f2', 'S7'), ('f3', '<f8'), ('f4', 'S7'), ('f5', '<f8'), ('f6', 'S7'), ('f7', '<f8'), ('f8', 'S7'), ('f9', '<f8'), ('f10', 'S5'), ('f11', 'S1'), ('f12', '<i4')])

add usecols

In [55]: dly_usecols = [1,7]
In [56]: data=np.genfromtxt(txt.splitlines(),dtype=None, comments=None,usecols=dly_usecols)
In [57]: data
Out[57]: 
array([(39584,  0.), (39585,  0.)],
      dtype=[('f0', '<i4'), ('f1', '<f8')])

same thing with the positional delimiter.

So that column delimiter does not override the comments. The code probably strips out comments before moving on to splitting the line with the delimiter.

genfromtxt returning -1 and nan rows

1 Answers1