Async db calls using recursion

Question

I have a need to recursively descend a linked list database tree

item 1-> item 2
      -> item 3 -> item 4
      -> item 5 -> item 6 -> item 7
      -> item 8

my pseudo code is

var getItems = function(itemid) {
    db.getitem(itemid, function(item) {

    item.items.forEach(function(subitem) {
        getItems(subitem.id)
    })
})

getItems(1)

however, db.getItem is an asynchronous function

I would like to return a JS object in the same structure as the diagram to the top-level caller

what is the best way of achieving this ? I don't know the structure up front (i.e. no idea on the number of items per item, or the depth of any branch in the tree) so I have no idea on the number of items I need to process

I have tried various methods of the async library, but none seem to deal with recursion

recursion depends on parent value i.e. caller will pass value, as you said your method is async, it is not the suggested way to do. you need to keep the operations as synchronized. — A.T., Apr 14 '15 at 09:35
And I assume you only need the children of a certain node, not all the tree all the time and the tree is large. Because if we're talking about 10-100 nodes then it would be faster/easier to just pull them all and build the tree "client-side" with JS. Maybe also cache the resulting tree? _Also_, there really is no way of doing true "synchronous" DB calls AFAIK(http://stackoverflow.com/questions/26839320/node-js-synchronous-database-call), except https://github.com/luciotato/waitfor but that's really against the whole point of JS and NodeJS. — Sergiu Paraschiv, Apr 14 '15 at 09:35
If it's a must, use Events (https://nodejs.org/api/events.html) instead of the async package. — Eduardo Yáñez Parareda, Apr 14 '15 at 09:42

score 3 · Accepted Answer · edited Apr 14 '15 at 14:17

This is where strong concurrency primitives shine.

Promises let you do this very easily:

// with bluebird this is var getItem = Promise.promisify(db.getitem);
var getItem = function(itemid){
     return new Promise(function(resolve, reject){
        db.getitem(itemid, function(err, data){
            if(err) reject(err);
            else resolve(data);
        });
     });
};

Which would let you do:

var getItems = function(itemid) {
    return getItem(itemid).then(function(item){ // get the first
       return Promise.all(item.items.forEach(function(subItem){
           return getItems(subitem.id);
       });
    }).then(function(subItems){
        var obj = {};
        obj[itemid] = subItems; // set the reference to subItems
        return obj; // return an object containing the relationship
    });
};


getItems(1).then(function(obj){
   // obj now contains the map as you describe in your problem description
});

Here is how it would look with async:

var getItems = function(itemid, callback){
   db.getitem(itemid, function(err, item){
       if(err) return callback(err, null);
       async.map(item.items, function(subitem, cb){
           getItems(subitem.id, cb);
       }, function(err, results){
           if(err) return callback(err, null);
           var obj = {};
           obj[itemid] = result;
           return callback(null, obj);
       });
   });
};

It gets pretty close but I think it's a lot less nice than the promise version.

damn, that's fine coding. Sorted my problem out perfectly. A million upvotes if I could ;) — jmls, Apr 14 '15 at 20:36

Async db calls using recursion

1 Answers1

Linked