Genfromtxt skips information with #

Question

I have a problem with reading in a CSV file with # signs. The CSV looks like this.

 aaa;;xxx;aaa;aaa;aaa;xxx;xxx;xxx;xxx;xxx;xxx;aaa

with aaa as a string and xxx as float. But in this file there is a line like this:

aaa;;aaa;#N/A;#N/A;#N/A;#N/A;#N/A;#N/A;#N/A;#N/A;#N/A;#N/A

Python keeps saying that this line would have 4 columns instead of 13. He interprets the # as a comment and skips the rest of it. I tried:

kwargs = dict(delimiter=';',
          dtype=np.str,
          skip_header=11,
          usecols= range(1,14),
          missing_values = "#N/A",
          filling_values = "0")
data = np.genfromtxt(TestFile, **kwargs)

but still couldn't get it to work.

How could I manage that?

What do you mean by read and evaluate the file. I've look through it and it contains all information which is necessary — Toggo, Nov 10 '17 at 13:55
Show us the code you use to read the file and extract each value from the CSV. — mrCarnivore, Nov 10 '17 at 13:56
'kwargs = dict(delimiter=';', dtype=np.str, skip_header=11, usecols= range(1,14), missing_values = "#N/A", filling_values = "0") data = np.genfromtxt(RigFile, **kwargs)' — Toggo, Nov 10 '17 at 14:01
Please edit your question and fill that in. Otherwise it is unreadable. — mrCarnivore, Nov 10 '17 at 14:06
You specified missing values as #N/A so no wonder it thinks that there are only 4 instead of 13 values. The other are missing according to your own definition. — mrCarnivore, Nov 10 '17 at 14:35
I may have missunderstood the genfromtxt, but wouldn't he fill in 0 if he find a #N/A? — Toggo, Nov 10 '17 at 14:43

score 0 · Accepted Answer · answered Nov 10 '17 at 14:35

0

Change the dictionary to,

kwargs = dict(delimiter=';',
              dtype=np.str,
              skip_header=11,
              usecols= range(1,14),
              missing_values = "#N/A",
              filling_values = "0",
              comments=None)

Now, this should work. However, I'm not sure why you're using columns 1-13 when there are only columns from 0-12.

answered Nov 10 '17 at 14:35

JerryTheo

91
5

Thank you for your answer. This works for me. I am reading columns 1-13 because the real csv has more columns... – Toggo Nov 10 '17 at 14:55

Genfromtxt skips information with #

1 Answers1