Numpy function to create arrays from tabular data.
Questions tagged [genfromtxt]
287 questions
5
votes
2 answers
Using genfromtxt to import csv data with missing values in numpy
I have a csv file that looks something like this (actual file has many more columns and rows):
1,2,3,4,5
6,7,8,9,10
11,12,13,14,15
16
Say the name of the file is info.csv
If I try to import this using
data = numpy.genfromtxt('info.csv', delimiter…
Curious2learn
- 31,692
- 43
- 108
- 125
5
votes
3 answers
numpy genfromtxt issues in Python3
I'm trying to use genfromtxt with Python3 to read a simple csv file containing strings and numbers. For example, something like (hereinafter "test.csv"):
1,a
2,b
3,c
with Python2, the following works well:
import…
Alessandro
- 371
- 3
- 11
5
votes
4 answers
Best approach to query SQL server for numpy
In a previous programme I was reading data from a csv file like this:
AllData = np.genfromtxt(open("PSECSkew.csv", "rb"),
delimiter=',',
dtype=[('CalibrationDate', datetime),('Expiry', datetime),…
Dan
- 45,079
- 17
- 88
- 157
4
votes
3 answers
How to read complex data in python?
I'm trying to read the data which is not structured well. It looks something like this
Generated by trjconv : P/L=1/400 t= 0.00000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 …
Mahesh
- 556
- 1
- 3
- 16
4
votes
1 answer
How to load csv file containing strings and numbers using genfromtxt?
I'm trying to load a csv file in a NumPy array for machine learning purpose. Until now I always worked with int or float data but my current csv contains string, float and int so I have some trouble with dtype argument. My datasets has 41188 samples…
pragma
- 41
- 1
- 3
4
votes
2 answers
numpy.genfromtxt- ValueError- Line # (got n columns instead of m)
So, I've been writing up code to read in a dataset from a file and separate it out for analysis.
The data in question is read from a .dat file, and looks like this:
14 HO2 O3 OH O2 O2
15 HO2 HO2 …
Tessa
- 79
- 1
- 11
4
votes
2 answers
Using multiple genfromtxt on a single file
I'm fairly new to Python and am currently having problems with handling my input file reads. Basically I want my code to take an input file, where the relevant info is contained in blocks of 4 lines. For my specific purpose, I only care about the…
AstroAT
- 482
- 1
- 6
- 16
4
votes
2 answers
Which is a good way to open a 'complicated' txt file in python
I have a txt file with the following format(simplified):
date this that other
2007-05-25 11:00:00 10 20 30
2007-05-25 11:10:00 15 18 30
2007-05-25 11:20:00 10 27 30
2007-05-25 11:30:00 20 35 30
2007-05-25…
Mpizos Dimitris
- 4,819
- 12
- 58
- 100
4
votes
2 answers
Is zip() the most efficient way to combine arrays with respect to memory in numpy?
I use numpy and have two arrays, which are read with genfromtxt.
They have the shape <10000,> according to np.shape().
I want these two vectors to be in an array with the shape <10000,2>. For now I use:
x = zip(x1,x2)
but i am not sure if there is…
user69453
- 1,279
- 1
- 17
- 43
4
votes
1 answer
numpy.genfromtxt seems to ignore dtype
I am trying to load a csv file consisting just from float types.
data = np.genfromtxt(self.file,dtype=float,delimiter=self.delimiter,names = True)
but this returns an array of tuples. Based on my search this should return tuples only for…
Pter
- 127
- 3
- 8
4
votes
4 answers
Importing data and variable names from a text file in Python
I have a text file containing simulation data (60 columns, 100k rows):
a b c
1 11 111
2 22 222
3 33 333
4 44 444
... where in the first row are variable names, and beneath (in columns) is the corresponding data (float type).
I need to use…
Michal
- 1,927
- 5
- 21
- 27
4
votes
1 answer
Skip row in genfromtxt
I have the following table:
2M00251602+5422547 7.180 9.000 2.200
#2M00255540+5749320 4.420 5.200 1.600
2M00274401+5330504 4.400 6.800 2.700
2M00331747+6327504…
Rohit
- 5,840
- 13
- 42
- 65
4
votes
3 answers
-9999 as missing value with numpy.genfromtxt()
Lets say I have a dumb text file with the contents:
Year Recon Observed
1505 162.38 23
1506 46.14 -9999
1507 147.49 -9999
-9999 is used to denote a missing value (don't ask).
So, I should be able to…
brews
- 661
- 9
- 23
3
votes
1 answer
genfromtxt dtype=None returns wrong shape
I'm a newcomer to numpy, and am having a hard time reading CSVs into a numpy array with genfromtxt.
I found a CSV file on the web that I'm using as an example. It's a mixture of floats and strings. It's here: http://pastebin.com/fMdRjRMv
I'm using…
Dan Evans
- 73
- 1
- 4
3
votes
2 answers
How to replace values when loading data with genfromtxt
I wonder how I can replace specific values when loading data from a given (csv) file with multiple columns, combining both strings and numerical values.
In the example that follows, suppose that you have a number of geographical positions, with…
gmaravel
- 357
- 1
- 3
- 12