Creating a Database and want to write out to disk and not keep in RAM

Question

Hi I am writing a Database in C and would like to write out data to Hard Drive and not have it stored in RAM anymore. Is there a way to do this?

Currently I create a big file and open it as such

fd = open("database.dat",O_CREAT | O_RDWR);
hd = mmap(0, SSD_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

And then write to memory addresses that have been mmapped and then

msync(dest,DB_PAGE_SIZE, MS_SYNC);

where dest in in the mmaped region.

DBMS like Oracle, Informix, DB2, PostgreSQL, Sybase, MySQL, MS SQL Server all manage it — I don't see why you can't. Of course, you have to design the system to work correctly, which isn't trivial, but it most certainly can be done. — Jonathan Leffler, Jul 28 '17 at 03:08

score 0 · Answer 1 · answered Jul 28 '17 at 08:11

mmap() just maps the file in the virtual address space, it does not load the whole file into the RAM. On 32 bit operating system your database will be quite limited with address space available, but on 64 bit operating system mmap() should be just fine.

So, assuming your OS is 64 bit, your solution if perfectly fine: operating system will manage the mapping so recently used pages will be in RAM and, in case of memory pressure, pages will be wrote back the disk automatically.

Creating a Database and want to write out to disk and not keep in RAM

1 Answers1