0

I have a list containing the strings of my data files. When I read the data I expect to see a numpy array with 3 columns of floats but instead I just get nan.

FILE_NAMES = ['file1.csv', 'file2.csv']
data = np.genfromtxt(FILE_NAMES, delimiter=',', skip_header=1)
print(data)

Here is an example of the data:

More of the same below, float values for all 3 columns

The values in all three of the columns are floats.

All it returns is:

nan

Can someone tell me what I need to change?

Jack
  • 69
  • 6
  • Use just one filename at a time, not a list of names. – hpaulj Dec 05 '21 at 22:18
  • @hpaulj yeah I have done that and it works fine but I am trying to adapt my code so it can work for many data files and all that is required is for the user to put these into the list rather than add any number of genfromtxt. – Jack Dec 05 '21 at 22:21
  • 1
    So you know how to use it correctly, and you want us explain why it doesn't work with wrong inputs! This the wrong forum for that. – hpaulj Dec 05 '21 at 22:31
  • @hpaulj I just wasn't sure why it worked when I used the files separately but gave me a different output when I used a list of strings which I assumed would give me the same output. I wasn't sure where I went wrong and I know it's something small but I could really use the help. Thank you. – Jack Dec 05 '21 at 22:37
  • Referring to [the documentation](https://numpy.org/doc/stable/user/basics.io.genfromtxt.html): *If a list of strings or a generator returning strings is provided, each string is treated as **one line in a file**.* So, they do not refer to the datafiles with that names. For doing so, I think, you must use `np.loadtxt` if the data permit. If not, you must use `np.genfromtxt` in loops to import them separately. – Ali_Sh Dec 05 '21 at 23:50
  • @Ali_Sh, `loadtxt` uses a list of strings in the same way. There's no shortcut to calling `genfromtxt` for each file. – hpaulj Dec 06 '21 at 01:18
  • @hpaulj You are right, I was wrong about `np.loadtxt` ability to handle multiple files at the same time. Thanks for correcting my mistake. – Ali_Sh Dec 06 '21 at 09:45

1 Answers1

1

That it didn't give an error initially puzzled me, but then I realized that:

genfromtxt accepts either a filename, or anything that gives it strings, like an open file, or a list of strings. I often use such a list with copy-n-paste file samples.

It's in effect parsing this file

file1.csv
file2.csv

With 1 header line and 1 non-float data line. Hence the nan data.

In [82]: alist = ['csv1.txt','csv2.txt','csv3.txt']
In [85]: data = np.genfromtxt(alist, dtype=None, encoding=True)
In [86]: data
Out[86]: array(['csv1.txt', 'csv2.txt', 'csv3.txt'], dtype='<U8')
hpaulj
  • 221,503
  • 14
  • 230
  • 353