1

I understand that the working set is cached every time a query is run in the mongoDB.

What happens if one working set exceeds the caching memory when I know that the data on the previous page is removed and cached?

Ex) cacheSizeGB: 0.5g, total document size about query: 1g

And is it helpful to reduce the size of the document being cached by using the project command?

1mm.p
  • 37
  • 5

1 Answers1

3

The cache is managed by WiredTiger, the mongod will request documents it needs, and if WT doesn't find it in the cache, it will read it in from disk. If this makes the cache exceed the threshold (default 80% of maximum size), the background eviction workers will start removing the least recently used items from the cache.

The result set is constructed/sorted in heap memory, not cache. If your result set is too large, mongod may be able to use disk for temporary space while sorting.

Large results may also cause the operating system to use swap, and in extreme cases it might run out of memory and get killed by the OOM killer.

Joe
  • 25,000
  • 3
  • 22
  • 44
  • I don't understand that "The result set is constructed/sorted in heap memory, not cache." Isn't the result cached every time? – 1mm.p Aug 22 '20 at 16:48
  • If the query is constructed so that no blocking operations, like in-memory sort, will be needed, the database can begin sending back results before it has reached the end of the cursor. The pages that were read from disk to build the result will be cached until evicted, but the result is not retained past the completion of the query. – Joe Aug 23 '20 at 17:54