Genfromtxt value error: got n columns instead 1

Question

I'm using numpy.genfromtxt to import a data column from a csv file as a variable twice (each column corresponding to different variables x and y). I'm getting a Value Error: (got 3 columns instead of 1) even though I;m specifying which column to use. The code is as follows:

import numpy as np
file = 'E:/TPA/AOM_8.csv'
file = os.path.normpath(file)
x = np.genfromtxt(file, delimiter=',', usecols=1, skip_header=9)
print(x)
y = np.genfromtxt(file, delimiter=',', usecols=3, skip_header=9)
print(y)

The weird thing is the code has no problem printing 'x' but it does with 'y'. If I change 'y' to

y = np.genfromtxt(file, delimiter=',', usecols=2, skip_header=9)

then it works.

A sample of the file is below:

ASCII data file created with TiePie Multi Channel software. www.tiepie.com.
2021-08-27 18:25:57 437m





Sample number,Relative time,Data collector1,Data collector2
 ,s,V,V
0,0,4.0176473,6.1250002
1,1E-5,4.0078434,6.0784316
2,2E-5,4.001961,6.1274512
3,3E-5,4.029412,6.0980394
4,4E-5,4.0333336,6.1274512
5,5E-5,4.0009806,6.1446081
6,6E-5,4.0205885,6.1274512
7,7E-5,4.0098042,6.1299022
8,8E-5,4.0058826,6.1078434
9,9E-5,4.0098042,6.154412
10,0.0001,4.0225493,6.1053924
11,0.00011,4.0323532,6.0588237
12,0.00012,4.0147061,6.1421571

maybe your skipheader is off. Did you examine the `x` values? Anything unexpected? — hpaulj, Aug 30 '21 at 15:29
@hpaulj I tried changing the number of lines to skip in the header and it did not solve the problem. From what I can tell 'x' looks correct. — Madmem, Aug 30 '21 at 16:00

score 0 · Answer 1 · answered Aug 30 '21 at 20:15

I don't have problems:

In [710]: txt='''ASCII data file created with TiePie Multi Channel software. www
     ...: .tiepie.com.
     ...: 2021-08-27 18:25:57 437m
     ...: 
     ...: 
     ...: 
     ...: 
     ...: 
     ...: Sample number,Relative time,Data collector1,Data collector2
     ...:  ,s,V,V
     ...: 0,0,4.0176473,6.1250002
     ...: 1,1E-5,4.0078434,6.0784316
     ...: 2,2E-5,4.001961,6.1274512
     ...: 3,3E-5,4.029412,6.0980394
     ...: 4,4E-5,4.0333336,6.1274512
     ...: 5,5E-5,4.0009806,6.1446081
     ...: 6,6E-5,4.0205885,6.1274512
     ...: 7,7E-5,4.0098042,6.1299022
     ...: 8,8E-5,4.0058826,6.1078434
     ...: 9,9E-5,4.0098042,6.154412
     ...: 10,0.0001,4.0225493,6.1053924
     ...: 11,0.00011,4.0323532,6.0588237
     ...: 12,0.00012,4.0147061,6.1421571'''
In [711]: txt=txt.splitlines()
In [712]: len(txt)
Out[712]: 22

all of it:

In [713]: data = np.genfromtxt(txt, delimiter=',', skip_header=9)
In [714]: data
Out[714]: 
array([[0.0000000e+00, 0.0000000e+00, 4.0176473e+00, 6.1250002e+00],
       [1.0000000e+00, 1.0000000e-05, 4.0078434e+00, 6.0784316e+00],
       [2.0000000e+00, 2.0000000e-05, 4.0019610e+00, 6.1274512e+00],
       [3.0000000e+00, 3.0000000e-05, 4.0294120e+00, 6.0980394e+00],
       [4.0000000e+00, 4.0000000e-05, 4.0333336e+00, 6.1274512e+00],
       [5.0000000e+00, 5.0000000e-05, 4.0009806e+00, 6.1446081e+00],
       [6.0000000e+00, 6.0000000e-05, 4.0205885e+00, 6.1274512e+00],
       [7.0000000e+00, 7.0000000e-05, 4.0098042e+00, 6.1299022e+00],
       [8.0000000e+00, 8.0000000e-05, 4.0058826e+00, 6.1078434e+00],
       [9.0000000e+00, 9.0000000e-05, 4.0098042e+00, 6.1544120e+00],
       [1.0000000e+01, 1.0000000e-04, 4.0225493e+00, 6.1053924e+00],
       [1.1000000e+01, 1.1000000e-04, 4.0323532e+00, 6.0588237e+00],
       [1.2000000e+01, 1.2000000e-04, 4.0147061e+00, 6.1421571e+00]])

single columns:

In [715]: x = np.genfromtxt(txt, delimiter=',', skip_header=9,usecols=1)
In [716]: x
Out[716]: 
array([0.0e+00, 1.0e-05, 2.0e-05, 3.0e-05, 4.0e-05, 5.0e-05, 6.0e-05,
       7.0e-05, 8.0e-05, 9.0e-05, 1.0e-04, 1.1e-04, 1.2e-04])
In [717]: x = np.genfromtxt(txt, delimiter=',', skip_header=9,usecols=2)
In [718]: x = np.genfromtxt(txt, delimiter=',', skip_header=9,usecols=3)
In [719]: x
Out[719]: 
array([6.1250002, 6.0784316, 6.1274512, 6.0980394, 6.1274512, 6.1446081,
       6.1274512, 6.1299022, 6.1078434, 6.154412 , 6.1053924, 6.0588237,
       6.1421571])

It has appeared to be a problem that comes and goes, and I haven't been able to figure out why. Thank you for trying anyways! — Madmem, Sep 01 '21 at 13:19
Thank you for your help, I was able to figure out the problem. The last row of the file didn't contain a value for each column. I didn't notice this for awhile because the file is so large (over a GB!) that when I would try to open the file in Notebook or Excel, only part of the file would load. Once I delete the last row, this error goes away. — Madmem, Jan 24 '23 at 19:46

Genfromtxt value error: got n columns instead 1

1 Answers1