Joining two dataframes on index creates null values

Asked Mar 05 '19 at 20:34

Active Mar 05 '19 at 20:34

Viewed 175 times

I have two dataframes which has non null values df1 and df2. Shape of df1 is (3681031, 52) and df2 is (3681031, 2). I'm concatenating the two dataframes using the following

df_merged = pd.concat([df1,df2], axis = 1)

I would expect the shape of the resulting dataframe to be (3681031, 54) however, it is (4667215, 54) and it also has a lot of nulls. Not sure whats going wrong as both the dataframes are sliced from the same dataframe.

I would appreciate any inputs.

asked Mar 05 '19 at 20:34

iprof0214

There might possibly be duplicates, etc. – cs95 Mar 05 '19 at 20:36
No they do not contain duplicates as the 2 columns in df2 are categorical and df1 contains only continuous variables. – iprof0214 Mar 05 '19 at 20:39
Tried adding ignore_index=True to the concat command? – AlexK Mar 05 '19 at 20:42
1

Also take a look at this [answer](https://stackoverflow.com/a/32802014/9987623) – AlexK Mar 05 '19 at 20:44
@AlexK : pd.concat([df1, df2], ignore_index=True) results in df of shape (7362062, 54) – iprof0214 Mar 05 '19 at 20:44
axis=1 should remain. And that link I provided should help you. – AlexK Mar 05 '19 at 20:48
@AlexK - that worked, thanks – iprof0214 Mar 05 '19 at 20:49

Joining two dataframes on index creates null values

0 Answers0