MongoDB allowDiskUse not working..

Question

Experts.

I'm new to MongoDB, but know enough to get my self in trouble.. case in point:

db.test.aggregate(
[
    {$group: {_id: {email: "$email", gender: "$gender"}, cnt: {$sum: 1}}}, 
    {$group: {_id: "$_id.email", cnt: {$sum: 1}}}, 
    {$match: {cnt: 2}}
], 
    {allowDiskUse : true}
)

and no matter what variations I try, I keep getting the same error ("Pass allowDiskUse:true to opt in"):

Error("Printing Stack Trace")@:0 ()@src/mongo/shell/utils.js:37 ([object Array],[object Object])@src/mongo/shell/collection.js:866 @(shell):7

uncaught exception: aggregate failed: { "errmsg" : "exception: Received error in response from mongo1.mscnet.com:27017: { $err: \"Exceeded memory limit for $group, but didn't allow external sort. Pass allowDiskUse:true to opt in.\", code: 16945 }", "code" : 16945, "ok" : 0 }

FYI: I'm using Mongo 2.6

Are you definitely using a 2.6 shell? What does `version()` return in the `mongo` shell? Also, are you using any shell extensions such as Mongo-Hacker? If so, I would try starting the shell using `mongo --norc` to test without extensions. Your syntax looks correct so I'm suspecting you may inadvertently be using a version of the `aggregate()` helper that does not support passing options to the aggregation command (which were added in the MongoDB 2.6 shell). — Stennie, Dec 16 '14 at 05:35
I'm using "MongoDB shell version: 2.6.5". Although, I was using a third-party app called RoboMongo (Version 0.8.4). I'm now using the CMD shell with --norc .. and that seems to work.. I guess it was my client. But can you explain something for me?.. The aggregation doesn't happen on my PC (does it?). I don't see any network / cpu usage on my pc. So what is the "client", and how/why does it effect the processing of data? Thank you! — Eyal Zinder, Dec 16 '14 at 19:48
Clients like the `mongo` shell or Robomongo provide [`aggregate()` helpers](http://docs.mongodb.org/manual/reference/method/db.collection.aggregate/) to make it more convenient to work with the underlying [`aggregate` command](http://docs.mongodb.org/manual/reference/command/aggregate/) that runs on your MongoDB server. In versions of MongoDB prior to 2.6 the aggregate command did not support any options so older versions of the `aggregate()` helpers will ignore those. If `--norc` works for the `mongo` shell I suspect you have an older version of Mongo-Hacker (I committed a fix in August 2014) — Stennie, Dec 16 '14 at 21:57
I see. So the client sends additional code (helpers) to the server that the server uses to build the execution-path and result - not just the actual command. Thank you again! — Eyal Zinder, Dec 16 '14 at 23:11
`mongo` shell helpers just manipulate parameters before sending the command to the server (for example, allowing you to pass an aggregation pipeline as either an array or a list of pipeline steps). FYI, you can see the helper code in the `mongo` shell by invoking without the (), eg: `db.test.aggregate` — Stennie, Dec 16 '14 at 23:28
Robomongo has not upgraded to the 2.6 shell and might not for some time. That's the issue. — neverfox, Feb 06 '15 at 20:54

score 21 · Answer 1 · answered Mar 20 '15 at 06:03

Please use aggregate query in run command,it will allow to use allowDiskUse tag.

db.runCommand(
   { aggregate: "test",
     pipeline: [
                {$group: {_id: {email: "$email", gender: "$gender"}, cnt: {$sum: 1}}}, 
                {$group: {_id: "$_id.email", cnt: {$sum: 1}}}, 
                {$match: {cnt: 2}}
               ],
     allowDiskUse: true
   }
)

score 16 · Answer 2 · edited Feb 08 '19 at 19:52

16

If you are working with m10 or later version of MongoDB Atlas, below is the fix for {allowDiskUse: true}; If you are accessing data from application via mongoose then, it should be syntax as function. For example:

MongoDb Query:

db.collection.user.aggregate([], {allowDiskUse: true})

With Mongoose:

User.aggregate([]).allowDiskUse(true);

This will resolve error from application :)

edited Feb 08 '19 at 19:52

Fcmam5

4,888
1
16
33

answered Feb 04 '19 at 15:21

pushpak

198
2
10

score 15 · Answer 3 · answered Apr 10 '17 at 05:10

Try writing like this,

db.stocks.aggregate([
                     { $project : { cusip: 1, date: 1, price: 1, _id: 0 } },
                     { $sort : { cusip : 1, date: 1 } }
                    ],
                     {
                       allowDiskUse: true
                     }
                    );

And One more thing,
Your error about maximum document size is inherent to Mongo. Mongo can never return a document (or array thereof) larger than 16 megabytes. I can't tell you why because you didn't mention anything about data size, but it probably means that the document you're building as an end result is too large. Try decreasing the $limit parameter, maybe? Start by setting it to 1, run a test, then increase it and look at how big the result gets when you do that

MongoDB allowDiskUse not working..

3 Answers3

Linked