I have been using R's filehash library to solve the "out of memory" problem, to store the large datasets in hashfiles, and the load/updating the file when use it. Given that most of the systems are now using SSD harddisk, I found this solution is good enough for me to solve my "out of memory" problem and a good balance between running time. I am now writing my codes in Python, but could not find any equivalent package. Could anyone shed some light for me? Thanks.
I have been reading some information about hashlib package in Python, but not quite sure if it is the samilar application as to create a hashfile and load it out.
