3

I am trying to figure out when are Mongodb indexes loaded in memory. Assuming I have n collections and each having m indexes. So when mongodb starts, will all n x m indexes be loaded in the memory?

As per the docs, they mention that if indexes fit in the RAM, all of them are loaded. If not, some of them are swapped to secondary storage. But I couldn't find a place where they have clarified that if on mongodb startup, are all indexes loaded?

This is important because it would allow us to get an estimate of how much RAM to expect for the db to function optimally.

PS: I am using aws-documentdb which I assume should have the similar behaviour for indexes as they also haven't touched this part in their docs anywhere.

Wan B.
  • 18,367
  • 4
  • 54
  • 71
rahulserver
  • 10,411
  • 24
  • 90
  • 164
  • 1
    DocumentDB is not based on the MongoDB server. Rather it only emulates the MongoDB API, and runs on top of Amazon’s Aurora backend platform. I think you should re-word your question. – Wan B. May 04 '20 at 05:20
  • Because if indexes are not in memory, performance becomes very bad – Basile Starynkevitch May 04 '20 at 05:25

1 Answers1

3

thank you for asking the question.

With most databases, including Amazon DocumentDB, index pages are paged into memory based on the queries that are run against the database (think of this as a lazy load). On start-up, the buffer cache is empty and will fill up with pages as your workload issues queries against the database. When an index(es) size is so big that it can't fit into memory, the database has to purge and read from disk to iterate through the index be able to response to a query. The same goes for data pages well. Ideally, you want to have enough RAM on your instance so that both your data pages and index pages fit in memory. Reads from disk will add additional latency. The best thing to do here is run your workload until it hits steady state and then observe the BufferCacheHitRatio to see if your queries are being served mainly from the buffer cache or if you're need to read from disk a lot. For more information, see: https://docs.aws.amazon.com/documentdb/latest/developerguide/best_practices.html

Joseph Idziorek
  • 4,853
  • 6
  • 23
  • 37