How to reverse a 2-dimensional table (DataFrame) into a 1 dimensional list using Pandas?

Question

I am looking in Python/Pandas for a tip that reverses a 2-dimension table into 1 dimensional list.

I usually leverage an Excel function to do it, but I believe that there is a smart Python way to do it.

Step

More details of the Excel way: http://www.extendoffice.com/documents/excel/2461-excel-reverse-pivot-table.html

Answer 1

This type of operation could also be done using pd.melt, which unpivots a DataFrame.

If the DataFrame df looks like this:

  row labels  Tue  Wed  Thu  Sat  Sun  Fri  Mon
0      Apple   21   39   24   27   37   46   42
1     Banana   32   50   48   35   21   27   22
2       Pear   37   20   45   45   31   50   32

Then we select the row_labels column to be our id_var and the rest of the columns to be our values (value_vars). We can even choose the new names for the columns at the same time:

>>> pd.melt(df, 
            id_vars='row labels', 
            value_vars=list(df.columns[1:]), # list of days of the week
            var_name='Column', 
            value_name='Sum of Value')

   row labels   Column   Sum of Value
0       Apple      Tue             21
1      Banana      Tue             32
2        Pear      Tue             37
3       Apple      Wed             39
4      Banana      Wed             50
5        Pear      Wed             20
...

The value_vars are stacked below each other: if the column values need to be in a particular order it will be necessary to sort the columns after melting.

Answer 2

This should do the trick:

table = [
            ["Lables", "A", "B", "C"],
            ["X", 1, 2, 3],
            ["Y", 4, 5, 6],
            ["Z", 7, 8, 9]
        ]

new_table = [["Row", "Column", "Data"]]
for line in table[1:]:
    for name, cell in zip(table[0], line)[1:]:
        new_line = [line[0], name, cell]
        new_table.append(new_line)

The output is:

[
    ['Row', 'Column', 'Data'],
    ['X', 'A', 1],
    ['X', 'B', 2],
    ['X', 'C', 3],
    ['Y', 'A', 4],
    ['Y', 'B', 5],
    ['Y', 'C', 6],
    ['Z', 'A', 7],
    ['Z', 'B', 8],
    ['Z', 'C', 9]
]

Answer 3

Example taken from http://pandas.pydata.org/pandas-docs/stable/reshaping.html

tl;dr, use:

from pandas import *
df.stack()

====================

Let's give an example of how this can be done.

Generate the sample data first:

from pandas import *
import pandas.util.testing as tm; tm.N = 3
import numpy as np
def unpivot(frame):
    N, K = frame.shape
    data = {'value' : frame.values.ravel('F'),
            'variable' : np.asarray(frame.columns).repeat(N),
            'date' : np.tile(np.asarray(frame.index), K)}
    return DataFrame(data, columns=['date', 'variable', 'value'])
df = unpivot(tm.makeTimeDataFrame())
df2=  df.pivot('date', 'variable')

We will unpivot this table:

               value                              
variable           A         B         C         D
date                                              
2000-01-03 -0.425081  0.163899 -0.216486 -0.266285
2000-01-04  0.078073  0.581277  0.103257 -0.338083
2000-01-05  0.721696 -1.311509 -0.379956  0.642527

Run:

df2=  df.pivot('date', 'variable')
print df2

Voila! Now we get the desired table.

                        value
date       variable          
2000-01-03 A        -0.425081
           B         0.163899
           C        -0.216486
           D        -0.266285
2000-01-04 A         0.078073
           B         0.581277
           C         0.103257
           D        -0.338083
2000-01-05 A         0.721696
           B        -1.311509
           C        -0.379956
           D         0.642527