5

Given these two dataframes, how do I get the intended output dataframe? The long way would be to loop through the rows of the dataframe with iloc and then use the map function after converting df2 to a dict to map the x and y to their score.

This seems tedious and would take long to run on a large dataframe. I'm hoping there's a cleaner solution.

df1:

ID    A    B    C
1     x    x    y
2     y    x    y
3     x    y    y

df2:

ID    score_x    score_y
1          20         30
2          15         17
3          18         22

output:

ID    A     B     C
1     20    20    30
2     17    15    17
3     18    22    22

Note: the dataframes would have many columns and there would be more than just x and y as categories (possibly in the region of 20 categories).

Thanks!

alwayscurious
  • 1,155
  • 1
  • 8
  • 18

3 Answers3

8

Use DataFrame.apply along columns with Series.map:

df1.set_index('ID', inplace=True)
df2.set_index('ID', inplace=True)
df2.columns = df2.columns.str.split('_').str[-1]

df1 = df1.apply(lambda x: x.map(df2.loc[x.name]), axis=1).reset_index()

print(df1)
   ID   A   B   C
0   1  20  20  30
1   2  17  15  17
2   3  18  22  22

print(df2)
     x   y
ID        
1   20  30
2   15  17
3   18  22
Space Impact
  • 13,085
  • 23
  • 48
4

Using mask:

df1.set_index('ID', inplace=True)
df2.set_index('ID', inplace=True)

df1.mask(df1=='x',df2['score_x'],axis=0).mask(df1=='y',df2['score_y'],axis=0)

Result:

     A   B   C
ID            
1   20  20  30
2   17  15  17
3   18  22  22

If there are many columns and they are all named in the same way, you can use something like that:

for e in df2.columns.str.split('_').str[-1]:
     df1.mask(df1==e, df2['score_'+e], axis=0, inplace=True)
Stef
  • 28,728
  • 2
  • 24
  • 52
  • 1
    @jezrael: see my addition to the answer, altough I admit it's not very elegant. – Stef Jul 10 '19 at 12:01
  • @jezrael Using `mask` for substituting 20 categories in a 1000 x 3 dataframe is about 10 times faster than using `apply` – Stef Jul 10 '19 at 12:47
0

There might be a more elegant way for this, but assuming you can enumerate through the categories and columns:

import numpy as np

df3 = df1.set_index('ID').join(df2.set_index('ID'), on='ID')
for col in ['A','B','C']:
     for type in ['x','y']:
         df3[col] = np.where(df3[col] == type, df3['score_'+type], df3[col])



>>> df3
     A   B   C  score_x  score_y
ID
1   20  20  30       20       30
2   17  15  17       15       17
3   18  22  22       18       22
Aditya Santoso
  • 1,031
  • 6
  • 19