3

I'm currently running in some issue with a collection of few hundreds of (heavy) documents.

Some of the fields of the document are pretty heavy, and we're excluding them during the projection stage.

We figure out that the sorting is actually done before the projection, resulting on some memory issues:

Overflow sort stage buffered data usage of 33658319 bytes exceeds internal limit of 33554432 bytes

Here are the questions:

  • Is it possible to manually specify the winning plan?
  • Is it possible to do the projection before sorting?
  • Why is sorting done before projecting?
Danziger
  • 19,628
  • 4
  • 53
  • 83
qventura
  • 68
  • 1
  • 6

1 Answers1

3

Is it possible to specify the winning plan?

I think you can't directly specify the winning plan, but you can use cursor.hint() to select an index to use in your query, which will influence the winning plan. For example, if you want to do a full collection scan, use { $natural : 1 }.

Keep in mind that as of MongoDB 2.6, hint() will be ignored if an index filter exists for your query shape (combination of query, sort and project).

Is it possible to project before sorting?

Yes, you can use the aggregation framework to specify the order of the operations and do the $project before the $sort.

Why is sort performed before projection?

If the sort is done before $project, $unwind, and $group, it can take advantage of indexes. Otherwise it can't. From the docs:

$sort Operator and Performance

$sort operator can take advantage of an index when placed at the beginning of the pipeline or placed before the $project, $unwind, and $group aggregation operators. If $project, $unwind, or $group occur prior to the $sort operation, $sort cannot use any indexes.

Community
  • 1
  • 1
Danziger
  • 19,628
  • 4
  • 53
  • 83