MongoDB $match $group query slow

Question

So I have the following query for about 500k documents and it is really slow on the database side:

db.collection('news').aggregate(
    [
      {
        $match: {
          feeds: { $in: feeds },
          createdAt: { $lte: lastPostDate },
          deleted: {
            $exists: false
          }
        }
      },
      {
        $group: {
          _id: { title: '$title' },
          id: { $first: '$_id' },
          title: { $addToSet: '$title' },
          source: { $first: '$source' },
          shortid: { $first: '$shortid' },
          stats: { $first: '$stats' },
          log: { $first: '$log' },
          createdAt: { $first: '$createdAt' },
          feedUpdated: { $first: '$feedUpdated' },
          media: { $first: '$media' },
          title: { $first: '$title' }
        }
      },
      { $sort: sort },
      {
        $skip: limit * (page - 1)
      },
      {
        $limit: limit * 1
      }
    ],
    {
      allowDiskUse: true,
      cursor: {}
    }
)

I know the allowDiskUse is making it slow, but if I disable it I get:

MongoError: Exceeded memory limit for $group, but didn't allow external sort. Pass allowDiskUse:true to opt in.

score 1 · Answer 1 · answered May 29 '20 at 13:05

let aggregation = MongoSchemaModel.aggregate([]); 
aggregation.options = { allowDiskUse: true }; 
aggregation.exec((err, data) => {});

OR

let aggregation = MongoSchemaModel.aggregate([], { allowDiskUse: true });
aggregation.exec((err, data) => {});

Reference:- https://mkyong.com/mongodb/mongodb-sort-exceeded-memory-limit-of-104857600-bytes/

MongoDB $match $group query slow

1 Answers1