$out stage of mongo aggregation pipeline not taking effect using node

Question

Long time listener, first time caller here.

I'm using node drivers to execute an aggregate command on mongo and the $out stage seems to only take effect if certain methods are chained.

I want to call aggregate() with $out as the last stage of the pipeline and then callback. Currently, it only works if I chain next() or toArray() which is irksome since the method signature is aggregate(pipeline, options, callback).

Examples below simplified for clarity.

Here, I use the callback and $out does NOT take effect (i.e. newNames isn't created but the callback is executed). I'm refactoring stuff from a few months back and this was working previously (version 2.2.33 of the node driver):

db.collection('names')
  .aggregate([
    { $match: {} }, 
    { $limit: 1 },
    { $out: 'newNames' }
  ], (err, result) => {
    // result is an AggregationCursor
    callback();
  });

If I don't add a callback but chain next() instead, the $out stage DOES take effect

db.collection('names')
  .aggregate([
    { $match: {} },
    { $limit: 1 },
    { $out: 'newNames' }
  ])
  .next((err, result) => {
    // result is null, why?
    callback();
  });

It also DOES work if you chain toArray:

db.collection('names')
  .aggregate([
    { $match: {} }, 
    { $limit: 1 },
    { $out: 'newNames' }
  ])
  .toArray((err, result) => {
    // result is an empty array (i.e. the resulting docs), OK, makes sense
    callback();
  });

So I thought I must be misunderstanding Promises vs callbacks, but it does NOT take effect if I use close() either which is chained and the result is back to being an AggregationCursor:

db.collection('names')
  .aggregate([
    { $match: {} },
    { $limit: 1 },
    { $out: 'newNames' }
  ])
  .close((err, result) => {
    // result is an AggregationCursor
    callback();
  });

Reading through some of the responses to issues, it seems the AggregationCursor vs Resulting Documents is expected. But I don't get why $out isn't taking effect when I go to the callback or chain close().

mongo 3.6.2 inside the mongo:latest docker image
Docker 17.12.0
node 8.9.0

score 7 · Answer 1 · answered Aug 24 '18 at 15:51

I just ran into this myself too. I called cursor.next() until a null was received and the $out aggregation worked. Using async / await makes this easy. Without these this could get pretty messy looking.

var cursor = await db.collection('names')
  .aggregate([
    { $match: {} },
    { $limit: 1 },
    { $out: 'newNames' }
  ]);

var next = null;
do {
   next = await cursor.next();
} while (next != null);

$out stage of mongo aggregation pipeline not taking effect using node

1 Answers1