What happens once a graph exceeds the RAM available to Memgraph?

Question

What happens once a graph exceeds the available RAM. Persistence is guaranteed through snapshots and WAL - but at some point, we will likely hit a limit on how much of the graph can be held in memory.

Is Memgraph aware of the completeness of the graph in memory? If so, does it have strategies to offload less-used paths from memory? And if that’s the case, how would it guarantee that a query returns a complete result (and might not have missed offloaded segments of the graph)? If file storage-based queries are part of the strategy, how much of a performance hit is to be expected?

Answer 1

Memgraph stores the entire graph inside RAM. In other words, there has to be enough memory to store the whole dataset. That was an early strategic decision because we didn’t want to sacrifice performance. At this point, there is no other option. Memgraph is durable because of snapshots and WALs + there are memory limits in place (at some point when the limit is reached, Memgraph will stop accepting writes). A side note, but essential to mention, is that graph algorithms usually use the entire dataset multiple times, which means you, either way, have to bring data to memory (Memgraph is really optimized for that case). There are drawbacks to the above approach. In the great majority of cases, you can find enough RAM, but it’s not the cheapest.