-1

I have a dictionary with 120 keys, each of which contains one data frame. How can I programmatically concatenate (using pd.concat) these into a single, large data frame?

scrollex
  • 2,575
  • 7
  • 24
  • 38

1 Answers1

1

Simply concatenate the values of your dictionary using pd.concat(d.values()):

>>> d = {1:pd.DataFrame(np.random.random((5,5))),2:pd.DataFrame(np.random.random((5,5)))}
>>> d
{1:           0         1         2         3         4
0  0.319556  0.540776  0.988554  0.775070  0.535067
1  0.383192  0.379474  0.204998  0.948605  0.785190
2  0.006732  0.362755  0.537260  0.854110  0.409386
3  0.795973  0.073652  0.796565  0.168206  0.814202
4  0.531702  0.524501  0.002366  0.631852  0.024509, 

2:           0         1         2         3         4
0  0.369098  0.125491  0.832362  0.183199  0.729110
1  0.069843  0.337424  0.476837  0.078589  0.489447
2  0.504904  0.456996  0.239802  0.025953  0.609697
3  0.262001  0.646389  0.992928  0.124552  0.878561
4  0.707881  0.520429  0.609257  0.018488  0.167053}

>>> new_df = pd.concat(d.values())
>>> new_df
          0         1         2         3         4
0  0.319556  0.540776  0.988554  0.775070  0.535067
1  0.383192  0.379474  0.204998  0.948605  0.785190
2  0.006732  0.362755  0.537260  0.854110  0.409386
3  0.795973  0.073652  0.796565  0.168206  0.814202
4  0.531702  0.524501  0.002366  0.631852  0.024509
0  0.369098  0.125491  0.832362  0.183199  0.729110
1  0.069843  0.337424  0.476837  0.078589  0.489447
2  0.504904  0.456996  0.239802  0.025953  0.609697
3  0.262001  0.646389  0.992928  0.124552  0.878561
4  0.707881  0.520429  0.609257  0.018488  0.167053
sacuL
  • 49,704
  • 8
  • 81
  • 106
  • Is there any difference between the answer above and adding each of the dfs in my dict to a list and then concatenating those? `list_of_dfs= [] for key in dict_of_dfs.keys(): list_of_dfs.append(dict_of_dfs[key]) df_overall = pd.concat(list_of_dfs)` – scrollex Nov 06 '18 at 17:42
  • 1
    I think the only difference would be that you are adding the creation of a list, which uses memory and time, both of which can be saved by *not* creating those lists – sacuL Nov 06 '18 at 17:43