numpy read txt error (loadtxt, genfromtxt)

Question

I would like to read text file like below in python

Example of text file

{#  "Column 4-5 Quantity": "Re/Im Pressure"
#   "Column 4-5 Units": ""
#   "Number Rows": 3584879
#}
4.0740740741e-002   -5.0370370370e-002  7.3333333333e-002   9.4443035889e+002   -6.0709484863e+002  
4.1481481481e-002   -5.0370370370e-002  7.3333333333e-002   9.0292657471e+002   -6.2604998779e+002  
4.2222222222e-002   -5.0370370370e-002  7.3333333333e-002   8.2029248047e+002   -6.4528021240e+002  
4.2962962963e-002   -5.0370370370e-002  7.3333333333e-002   7.1829382324e+002   -6.5028546143e+002  
4.3703703704e-002   -5.0370370370e-002  7.3333333333e-002   1.#QNAN00000e+000   0.0000000000e+000   
4.4444444444e-002   -5.0370370370e-002  7.3333333333e-002   1.#QNAN00000e+000   0.0000000000e+000   
4.5185185185e-002   -5.0370370370e-002  7.3333333333e-002   1.#QNAN00000e+000   0.0000000000e+000   
4.5925925926e-002   -5.0370370370e-002  7.3333333333e-002   1.#QNAN00000e+000   0.0000000000e+000
-3.2592592593e-002  -4.9629629630e-002  7.3333333333e-002   4.9716027832e+002   -1.2688856201e+003  
-3.1851851852e-002  -4.9629629630e-002  7.3333333333e-002   5.3677227783e+002   -1.4196939697e+003  
-3.1111111111e-002  -4.9629629630e-002  7.3333333333e-002   6.1588317871e+002   -1.5679577637e+003  
-3.0370370370e-002  -4.9629629630e-002  7.3333333333e-002   7.2673925781e+002   -1.7039268799e+003  
-2.9629629630e-002  -4.9629629630e-002  7.3333333333e-002   8.5946520996e+002   -1.7935989990e+003  
-2.8888888889e-002  -4.9629629630e-002  7.3333333333e-002   1.0030370483e+003   -1.7982950439e+003

Thus I used numpy's function loadtxt and genfromtxt

First I tried loadtxt the code is

impoty numpy as np

readtxt = np.loadtxt("filename.txt")

and I got error message like below

Wrong number of columns at line 5

Second I tried genfromtxt the code is

impoty numpy as np

readtxt = np.genfromtxt("filename.txt")

and I got error message like below

Line #5 (got 5 columns instead of 4)

So, How could I figure out this problem? I think the problem is "1.#QNAN00000e+000"

And I would like to change this value as "0"

The initial '{#' is a problem. It's one data column followed be comment. — hpaulj, Mar 02 '18 at 12:29

Paul Panzer · Accepted Answer · 2018-03-02T08:56:40.860

You can use the comments and missing_values kwds. comments we need to set to something other than its default # so the parser doesn't stop reading in the middle of the line. missing values we set to '1.#QNAN0000e+000', so they are handled gracefully. These are set to nan but the nans you can easily replace with zeros afterwards (result[np.isnan(result)] = 0):

>>> np.genfromtxt(<your_file>, comments='%', missing_values='1.#QNAN0000e+000')
array([[ 4.07407407e-02, -5.03703704e-02,  7.33333333e-02,
         9.44430359e+02, -6.07094849e+02],
       [ 4.14814815e-02, -5.03703704e-02,  7.33333333e-02,
         9.02926575e+02, -6.26049988e+02],
       [ 4.22222222e-02, -5.03703704e-02,  7.33333333e-02,
         8.20292480e+02, -6.45280212e+02],
       [ 4.29629630e-02, -5.03703704e-02,  7.33333333e-02,
         7.18293823e+02, -6.50285461e+02],
       [ 4.37037037e-02, -5.03703704e-02,  7.33333333e-02,
                    nan,  0.00000000e+00],
       [ 4.44444444e-02, -5.03703704e-02,  7.33333333e-02,
                    nan,  0.00000000e+00],
       [ 4.51851852e-02, -5.03703704e-02,  7.33333333e-02,
                    nan,  0.00000000e+00],
       [ 4.59259259e-02, -5.03703704e-02,  7.33333333e-02,
                    nan,  0.00000000e+00],
       [-3.25925926e-02, -4.96296296e-02,  7.33333333e-02,
         4.97160278e+02, -1.26888562e+03],
       [-3.18518519e-02, -4.96296296e-02,  7.33333333e-02,
         5.36772278e+02, -1.41969397e+03],
       [-3.11111111e-02, -4.96296296e-02,  7.33333333e-02,
         6.15883179e+02, -1.56795776e+03],
       [-3.03703704e-02, -4.96296296e-02,  7.33333333e-02,
         7.26739258e+02, -1.70392688e+03],
       [-2.96296296e-02, -4.96296296e-02,  7.33333333e-02,
         8.59465210e+02, -1.79359900e+03],
       [-2.88888889e-02, -4.96296296e-02,  7.33333333e-02,
         1.00303705e+03, -1.79829504e+03]])

It still make same errors ,, I think it because comments lines in text file. I edit my example text file above ,, — 박태영, Mar 02 '18 at 09:08
@박태영 If these comments are all at the top you can just skip them using `skip_header=n` wheree n is the number of lines you want ignored. — Paul Panzer, Mar 02 '18 at 09:12

numpy read txt error (loadtxt, genfromtxt)

1 Answers1