Copy values only to a new empty dataframe with column names - Pandas

Question

I have two dataframes.

df1= pd.DataFrame({'person_id':[1,2,3],'gender': ['Male','Female','Not disclosed'],'ethnicity': ['Chinese','Indian','European']})
df2 = pd.DataFrame(columns=['pid','gen','ethn'])

As you can see, the second dataframe (df2) is empty. But may also contain few rows of data at times

What I would like to do is copy dataframe values (only) from df1 to df2 with column names of df2 remain unchanged.

I tried the below but both didn't work

df2 = df1.copy(deep=False)
df2 = df1.copy(deep=True)

How can I achieve my output to be like this? Note that I don't want the column names of df1. I only want the data

No, I can't use rename. Because there are several dataframes coming in, don't think it would be best to rename one by one — The Great, Jul 29 '19 at 11:16

Ankur Sinha · Accepted Answer · 2019-07-29T11:29:11.043

3

Do:

df1.columns = df2.columns.tolist()
df2 = df2.append(df1)

## OR 

df2 = pd.concat([df1, df2])

Output:

  pid            gen      ethn
0   1           Male   Chinese
1   2         Female    Indian
2   3  Not disclosed  European

Edit based on OPs comment linking to the nature of dataframes:

df1= pd.DataFrame({'person_id':[1,2,3],'gender': ['Male','Female','Not disclosed'],'ethn': ['Chinese','Indian','European']})
df2= pd.DataFrame({'pers_id':[4,5,6],'gen': ['Male','Female','Not disclosed'],'ethnicity': ['Chinese','Indian','European']})
df3= pd.DataFrame({'son_id':[7,8,9],'sex': ['Male','Female','Not disclosed'],'ethnici': ['Chinese','Indian','European']})
final_df = pd.DataFrame(columns=['pid','gen','ethn'])

Now do:

frame = [df1, df2, df3]

for i in range(len(frame)):
    frame[i].columns = final_df.columns.tolist()
    final_df = final_df.append(frame[i])

print(final_df)

Output:

  pid            gen      ethn
0   1           Male   Chinese
1   2         Female    Indian
2   3  Not disclosed  European
0   4           Male   Chinese
1   5         Female    Indian
2   6  Not disclosed  European
0   7           Male   Chinese
1   8         Female    Indian
2   9  Not disclosed  European

edited Jul 29 '19 at 11:29

answered Jul 29 '19 at 11:16

Ankur Sinha

6,473
7
42
73

can't we use `concat` as well to do this task? – Rajan Sharma Jul 29 '19 at 11:19
For concat, columns names doesn't matter? – The Great Jul 29 '19 at 11:21
I mean I have 5 dataframes with different columns. I would like to have one final dataframe (which is already created with columns but empty) with all these 5 df put vtogether – The Great Jul 29 '19 at 11:22
To my knowledge, it will matter, else you will end up with 3 extra columns with NaN values because it will not know how to concatenate. – Ankur Sinha Jul 29 '19 at 11:23
May be you can edit your question and make it broader. The different columns that you have perhaps. This one works for your given scenario but looks like you want something beyond it. Unless one sees, I suppose it will be difficult to come help. – Ankur Sinha Jul 29 '19 at 11:24
https://stackoverflow.com/questions/57250943/append-dataframes-with-different-column-names-pandas/57252662#57252662 – The Great Jul 29 '19 at 11:24
Please check now from the Edit part and the result of final_df! Guess now it should work for you. – Ankur Sinha Jul 29 '19 at 11:29
Can you help me with this post? https://stackoverflow.com/questions/57285680/dont-drop-all-nas-during-stack-operation-in-python – The Great Jul 31 '19 at 08:15

score 2 · Answer 2 · answered Jul 29 '19 at 11:26

2

The cleanest solution I think is to just append the df1 after its column names have been set properly:

df2 = df2.append(pd.DataFrame(df1.values, columns=df2.columns))

answered Jul 29 '19 at 11:26

Stef

28,728
2
24
52

Any reason why your removed your answer from my another post? I was thinking it was easy to understand. Any issues with it? – The Great Jul 30 '19 at 07:59

Copy values only to a new empty dataframe with column names - Pandas

2 Answers2