Performance difference of Boost r-tree in memory vs. mapped file

Question

I need to create an 3D R*-tree, perhaps for long time storage, but performance will also be an issue. In order to create the tree, I decided to use Boost's spacialindex and basically found two possible methods.

Either I create it directly using objects as it's here: Index of polygons stored in vector, however that does not allow me to store and load it without creating the R*-tree again.

Or I could use a mapped file as explained here: Index stored in mapped file using Boost.Interprocess, however, I am not sure if the performance of queries is good enough in this case.

My r-tree will contain several thousand entries, but most likely less than about 100,000. Now my question is, is there any strong performance issue by using mapped files compared to using the standard objects? Also, if the creation of an R*-tree of about 100,000 values does not take a substanial amount of time (I could have all bounding boxes and corresponding keys/data stored in a file) then it might be a better option to skip the mapped file and just create the tree every time I run the program?

Hopefully, somebody can help me here, as the documentation does not really provide much information (though it's still worlds better than the documentation of libspacialindex).

Answer 1

A mapped file will behave mostly like regular memory (in fact, in Linux, memory allocation with new or malloc will use mmap [with a "no file" backing storage] as the underlying allocation method). However, if you do many small writes "all over the place", and you are mapping over a REAL FILE, then the OS will restrict the amount of buffered writes before writing to the file.

I did some experiments when the subject came up a while ago, and by adjusting the settings for how the OS deals with these "pending writes", I got reasonably performance even for filebacked memory mapping with random read/write pattern [something I expect happens when you are building your tree].

Here's the "performance of mmap with random writes" question, which I think is highly related: Bad Linux Memory Mapped File Performance with Random Access C++ & Python (This answer applies to Linux - other OS's, in particular Windows, may well behave completely differently with regards to how it deals with writes to mapped files)

Of course, it's pretty hard to say "which is better", between memory mapped file or rebuild every time the program is run - it really depends on what your application does, whether you run it 100 times a second, or once a day, how long it takes to rebuild [I have absolutely no idea!], and lots of other such things. There are two choices: Build the simplest version, and see if it's "fast enough", or build both versions, and measure how much difference there is, and then decide which path to go down.

I tend to build the simple(ish) model, and if performance isn't good enough, figure out where the slowness comes from, and then fix that - it saves spending lots of time making something that takes 0.01% of the total execution time run 5 clock-cycles faster, and ending up with a big thinko somewhere else that makes it run 500 times slower than you expected...

Answer 2

Bulk-loading the index is much faster than repeated insertion, and yields a much more efficient tree. So if you can hold all your data in main memory, I suggest rebuilding the tree using STR bulk loading. In my experience this is more than fast enough (bulk loading time is dwarfed by I/O time).

Cost of STR is roughly that of sorting. O(n log n) theoretically, with very low constants (a less efficient implementation may be O(n log n log n) but that still is fairly cheap).