Retriving all the rows from a csv file and plotting

Question

I need to retrieve the rows from a csv file generated from the function:

def your_func(row):
    return (row['x-momentum']**2+ row['y-momentum']**2 + row['z-momentum']**2)**0.5 / row['mass']

columns_to_keep = ['#time', 'x-momentum', 'y-momentum', 'z-momentum', 'mass']
dataframe = pd.read_csv("./flash.csv", usecols=columns_to_keep)
dataframe['mean_velocity'] = dataframe.apply(your_func, axis=1)

print dataframe

I got rows up until 29s then it skipped to the last few lines, also I need to plot this column 2 against 1

If you mean the problem is that it displays only the first lines then `...` then the last lines, it's just how pandas display dataframes, instead of filling screen with numbers. — polku, Jun 22 '16 at 16:00
but can I retrive all the rows, what if I want the whole data to make plots? — bhjghjh, Jun 22 '16 at 16:30
Ok maybe I'm not explicit enough, check the row count of your df against the lines of your csv : you HAVE all the data. Pandas is a library to manipulate data, not to do nice display. Print is useful only to check everything looks like expected, and filling the screen with numbers when you have a big dataframe is not a good way to do that, so pandas do it differently. — polku, Jun 22 '16 at 17:36
OK, I understand, then may I look for a particular row with a speciall command, suppose I want to check the data in row 497, and that's hidden within the dots(...), is there any special way to see it? — bhjghjh, Jun 22 '16 at 17:56
Asking this question means you still don't understand. Data is different from representation of data. I don't know how to explain it differently, but from what I remember pandas have a pretty good doc. — polku, Jun 22 '16 at 18:19
Now I get it and I could plot my data too, thanks for your explanation — bhjghjh, Jun 23 '16 at 06:13
Go back to your other question. [As I have pointed out there](http://stackoverflow.com/questions/37956344/reading-and-doing-calculation-from-dat-file-in-python#comment63415164_37956460), the code you accepted as a solution is wrong. You might be using completely bogus data. — Jan Christoph Terasa, Jun 23 '16 at 08:20
@Christoph, I plotted the data with your algorithm, these actually match with my previous data collected from the other code, thanks anyway, learnt a lot thing in past few days — bhjghjh, Jun 24 '16 at 07:25
@bhjghjh, it works because you apparently only use the first few columns. As soon as you want to use one of the latter ones with whitespace in them it will fail. — Jan Christoph Terasa, Jun 24 '16 at 08:10

score 1 · Accepted Answer · answered Jun 22 '16 at 17:32

you can adjust pd.options.display.max_rows option, but it won't affect your plots, so your plots will contain all your data

demo:

In [25]: df = pd.DataFrame(np.random.randint(0,100,size=(10, 3)), columns=list('ABC'))

In [26]: df
Out[26]:
    A   B   C
0  93  76   5
1  33  70  12
2  50  52  26
3  88  98  85
4  90  93  92
5  66  10  58
6  82  43  39
7  17  20  91
8  47  90  33
9  44  30  26

In [27]: pd.options.display.max_rows = 4

Now it'll display 4 rows at most

In [36]: df
Out[36]:
     A   B   C
0   93  76   5
1   33  70  12
..  ..  ..  ..
8   47  90  33
9   44  30  26

[10 rows x 3 columns]

but it'll plot all your data

In [37]: df.plot.bar()
Out[37]: <matplotlib.axes._subplots.AxesSubplot at 0x49e2d68>

In [38]: pd.options.display.max_rows = 60

In [39]: df
Out[39]:
    A   B   C
0  93  76   5
1  33  70  12
2  50  52  26
3  88  98  85
4  90  93  92
5  66  10  58
6  82  43  39
7  17  20  91
8  47  90  33
9  44  30  26

Retriving all the rows from a csv file and plotting

1 Answers1