R dcast equivalent in python pandas

Question

I am trying to do the equivalent of the below commands in python:

test <- data.frame(convert_me=c('Convert1','Convert2','Convert3'),
                   values=rnorm(3,45, 12), age_col=c('23','33','44'))
test

library(reshape2)
t <- dcast(test, values ~ convert_me+age_col, length  )
t

That is, this:

convert_me   values     age_col
Convert1     21.71502      23
Convert2     58.35506      33
Convert3     60.41639      44

becomes this:

values     Convert2_33 Convert1_23 Convert3_44
21.71502          0           1           0
58.35506          1           0           0
60.41639          0           0           1

I know that with dummy variables I can get the value of the columns and transform as the name of the column, but is there a way to merge them(combination) easily, as R does?

What's the higher-level reason you want to do this particular transformation? — John Zwinck, Sep 02 '14 at 08:09
I am creating columns so that I can use it in machine learning algorithms. — Adriano Almeida, Sep 02 '14 at 08:14
Do you realize that what R is giving you has a number of columns which is `1 + length(convert_me) * length(age_col)`? At first I thought you would want one column per unique age, but that's not what R is doing for you (you can see if you assign the same age to two rows). — John Zwinck, Sep 02 '14 at 08:37

joris · Accepted Answer · 2014-09-02T08:35:40.817

You can use the crosstab function for this:

In [14]: pd.crosstab(index=df['values'], columns=[df['convert_me'], df['age_col']])
Out[14]: 
convert_me  Convert1  Convert2  Convert3
age_col           23        33        44
values                                  
21.71502           1         0         0
58.35506           0         1         0
60.41639           0         0         1

or the pivot_table (with len as the aggregating function, but here you have to fillna the NaNs with zeros manually):

In [18]: df.pivot_table(index=['values'], columns=['age_col', 'convert_me'], aggfunc=len).fillna(0)
Out[18]: 
age_col           23        33        44
convert_me  Convert1  Convert2  Convert3
values                                  
21.71502           1         0         0
58.35506           0         1         0
60.41639           0         0         1

See here for the docs on this: http://pandas.pydata.org/pandas-docs/stable/reshaping.html#pivot-tables-and-cross-tabulations

Most functions in pandas will return a multi-level (hierarchical) index, in this case for the columns. If you want to 'melt' this into one level like in R you can do:

In [15]: df_cross = pd.crosstab(index=df['values'], columns=[df['convert_me'], df['age_col']])

In [16]: df_cross.columns = ["{0}_{1}".format(l1, l2) for l1, l2 in df_cross.columns]

In [17]: df_cross
Out[17]: 
          Convert1_23  Convert2_33  Convert3_44
values                                         
21.71502            1            0            0
58.35506            0            1            0
60.41639            0            0            1

thats almost that. Is there a way to merge the convert_me and age_col? How can I create a single table, without the levels. — Adriano Almeida, Sep 02 '14 at 08:36

score 3 · Answer 2 · answered Feb 22 '18 at 02:44

We can use pd.get_dummies function. In the current pandas 0.22.0, it is common to use pd.get_dummies when one-hot encoding to Dataframe.

import pandas as pd

df_dummies = pd.get_dummies(
    df[['convert_me', 'age_col']].apply(lambda x: '_'.join(x.astype(str)), axis=1),
    prefix_sep='')
df = pd.concat([df["values"], df_dummies], axis=1)
# Out[39]:
#      values  Convert1_23  Convert2_33  Convert3_44
# 0  21.71502            1            0            0
# 1  58.35506            0            1            0
# 2  60.41639            0            0            1

R dcast equivalent in python pandas

2 Answers2

Linked