0

I have a table having duplicate rows in consecutive rows. Row having same 'id' should have duplicate data in other columns.But there are few rows in which data is not proper. Eg - 

id  Name    Age
1   Ram     12
1   Ram     10
2   Shyam   11
2   Yam     11
3   Ravi    23
3   Ravi    23
4   Harsh   34
4   Harsh   34

I need to know the columns in which the columns differ for consecutive rows.

Final output I need - 

id  Name    Age     DifferentColumn
1   Ram     12      
1   Ram     10      Age
2   Shyam   11
2   Yam     11      Name
3   Ravi    23 
3   Ravi    23
4   Harsh   34
4   Krish   54      Name,Age

I can use 'petl' or 'pandas' for this but what should be my approach?

TeeKay
  • 1,025
  • 2
  • 22
  • 60

1 Answers1

1

okay this works

   id   Name  Age
0   1    Ram   12
1   1    Ram   10
2   2  Shyam   11
3   2    Yam   11
4   3   Ravi   23
5   3   Ravi   23
6   4  Harsh   34
7   4  Krish   54

df['Match'] = df.groupby('id').apply(lambda x: [' ','Name,Age'] if ((len(set(x.Name)) > 1) and (len(set(x.Age)) > 1)) else [' ','Age'] if len(set(x.Age)) > 1 else [' ','Name'] if ((len(set(x.Name)) > 1)) else [' ',' ']).reset_index(name='Match').apply(lambda x: pd.Series(x.Match), axis=1).stack().reset_index(drop=True)

Whats going on

pd.groupby by id, then straight if conditions to verify where the name and age are different or same.Steps create something like below

   id          Match
0   1       [ , Age]
1   2      [ , Name]
2   3         [ ,  ]
3   4  [ , Name,Age]

Next, just open the list and stack them up.

Output

  id   Name  Age     Match
0   1    Ram   12          
1   1    Ram   10       Age
2   2  Shyam   11          
3   2    Yam   11      Name
4   3   Ravi   23          
5   3   Ravi   23          
6   4  Harsh   34          
7   4  Krish   54  Name,Age
iamklaus
  • 3,720
  • 2
  • 12
  • 21