1

I am trying to merge a couple of DataFrames using pandas merge (by left index) and it fails because the machine is running out of RAM.

What are some disk based alternatives to achieve the same results with minimal changes to code?

user2808117
  • 4,497
  • 8
  • 27
  • 36
  • 1
    see this question: http://stackoverflow.com/questions/14614512/merging-two-tables-with-millions-of-rows-in-python/14617925#14617925, and using HDFStore: http://pandas.pydata.org/pandas-docs/dev/cookbook.html#hdfstore – Jeff Nov 26 '13 at 13:47
  • 1
    Hey @Jeff, this worked great. I just have one comment for future readers which I couldn't add to your referenced solution - the merge function would probably need to change, meaning a left join between two 'full' dataframes would turn into an inner join, since not all indices would be available for each 'small' merge. Hope I managed to explain myself clearly and that this would help. Thanks again. – user2808117 Nov 27 '13 at 10:24
  • updated the prior answer..thxs – Jeff Nov 27 '13 at 14:27

0 Answers0