Async map with multiple insert inside

Question

I need to insert inside a MongoDB a list of topics with unique slugs.

This is an example of two topics:

{
  title: "my title"
},
{
  title: "my title"
}

I then need to generate the slugs and insert my topics:

// For each topic perform insert
async.map(topics, function (topic, done) {

    // Generate unique slug
    topic.slug = slug(topic.title).toLowerCase();

    // Look into DB to find topic with same slug
    self._collection.findOne({slug: topic.slug}, function(err, result) {

        // If there is a result then generate an unique slug prepending a shortId to the slug
        if (result) {
            topic.slug = shortId.generate() + "-" + topic.slug;
        }

        // Insert new topic into db
        self._collection.insert(topic, function (err, docs) {
            return done(err, docs[0]);
        });
    });

}, done);

My problem is that being async, the finds are done all together and so they don't find anything because the inserts are performed all together before the finds.

How can I fix this problem without loose the advantage of async operations?

Answer 1

All you really seem to need to do is change this to async.mapSeries:

// For each topic perform insert
async.mapSeries(topics, function (topic, done) {

    // Generate unique slug
    topic.slug = slug(topic.title).toLowerCase();

    // Look into DB to find topic with same slug
    self._collection.findOne({slug: topic.slug}, function(err, result) {

        // If there is a result then generate an unique slug prepending a shortId to the slug
        if (result) {
            topic.slug = shortId.generate() + "-" + topic.slug;
        }

        // Insert new topic into db
        self._collection.insert(topic, function (err, docs) {
            return done(err, docs[0]);
        });
    });

}, done);

Acutally, since you are not really altering the array then .eachSeries() is really what you want.

The "series" types in the async library all wait for the current operations or "iteration" in this case to complete, as the callback is then used to signal the end before the other starts.

Of course, these loops make use of events and do not block.