How do I read records from MapDB in batches, instead of reading all at once ?
I have a list of say 1,000 records that I have persisted on disk through MapDB.
private fun storeRecordsInFile(records : List<Record>): String {
val fileName = "/some/location/temp_file.db"
val db = DBMaker.fileDB(fileName).fileMmapEnable().closeOnJvmShutdown().make()
val diskBasedRecordsList = db.indexTreeList("recordsList").create()
records.forEach { diskBasedRecordsList.add(it) }
db.commit()
db.close()
return fileName
}
I am able to read all of these records at once.
private fun readRecordsFromFile(fileName: String): List<Record> {
val db = DBMaker.fileDB(fileName).fileMmapEnable().make()
val diskBasedRecordsList = db.indexTreeList("recordsList").open()
//read from disk based list
val recordsList = mutableListOf<Record>()
val size = diskBasedRecordsList.size
for(i in 0 until size-1){
recordsList.add(diskBasedRecordsList[i] as Record)
}
db.close()
return recordsList
}
However, I would want to read them in a chunk/batch of only 100 at a time in memory (i.e. without fetching the entire 1000 records in memory at once) ?
