Node.js - Ensuring a non-blocking call finishes

Question

I'm using the pg postgres node module to interact with my database. The code works without an issue when I use pg's evented API. Below is an example of the code:

Migration.js

 exports.up = function(logger){
    var pg = require("pg")
        , connectionString = //process.env.CONNECTIONSTRING
        , client = new pg.Client(connectionString);

    client.connect();

    var cmd = "CREATE TABLE users( "
                + "id SERIAL NOT NULL, "
                + "firstName VARCHAR(50) NOT NULL, "
                + "lastName VARCHAR(50) NOT NULL, "
                + "CONSTRAINT pk_userid PRIMARY KEY (id) "
                + ")";

    var query = client.query(cmd);

    query.on("end", function(){
        client.end();
            logger.log("complete");
    });
};

I'm using commander.js to write a command line utility around this migration script; however, the non-blocking call of the pg snippet is freeing up the command line interface script to finish before the database updates have been done. Below is an example commander snippet:

MyCliTool

var program = require('commander');

program
    .version('1.0.2');

program
    .command("up")
    .description("Migrates up")
    .action(function(){
        require("../src/migration").up();
    });

// Additional code removed for brevity

Is there a way that I can change the migration (or the commander.js app) script to ensure that the migration's up() function will finish prior to my cli script finishing? I have tried using callbacks but that appears to not be working.

UPDATE Another example to illustrate this point. Below is a unit test (written with mocha) concerning the issue.

Up-test.js

describe("feature", function(){
    it("should finish", function(){
        var logger = {};
        var finished = false;
        logger.log = function(){finished = true;};

        var migration = require("../src/migration.js");
        migration.up(logger);
        assert(finished, "function didn't finish"); // finished is false when this gets called.
    });
});

Have you tried passing the callback to client.query() instead of using query.on()? — joshuapoehls, Jul 06 '12 at 21:04
I've tried `pg`'s callback syntax, passing in a callback at the end as well as adding a callback to the `up()` function and calling in after `client.end()` and in both cases, the code that calls the `up()` function continues on, past the call and ultimately finishes before `up()` does. — JamesEggers, Jul 06 '12 at 21:08
That is the way asynchronous code functions. If you want to "wait" for up to continue, you need to have up() call a callback and only continue execution from that callback. — Jakob Borg, Jul 06 '12 at 21:23
Your up-test.js looks like it has a bit of a bug: `assert(finished);` should be inside of your `logger.log` function, or inside of something that won't be executed immediately. Mocha supplies you a callback for async tests which you'll have to use as well: `it("should finish", function (completedCallback) { });` see http://visionmedia.github.com/mocha/#asynchronous-code — Andrew Dunkman, Jul 06 '12 at 22:01

score 3 · Answer 1 · answered Jul 06 '12 at 21:57

Looks like your original script has a pain point around program.action(function () {});:

program
    .command("up")
    .description("Migrates up")
    .action(function(){
        require("../src/migration").up();

        // nothing preventing this function from exiting immediately
    });

I poked around a bit in commander.js docs and couldn't find anything regarding supplying a callback to your .action() program. If I were writing it, the code would look something like this:

program
    .command("up")
    .description("Migrates up")
    .action(function (completedCallback) {

        require("../src/migration").up(function () {
            // migration is complete
            completedCallback();
        });

    });

Your postgre-related script looks fine, though. A somewhat unrelated note might be to have require("../src/migration").up(); return an instance of require("events").EventEmitter and emit events you'd like to have visibility into, instead of using a logger variable. But that's just preference.

score 0 · Answer 2 · answered Jul 07 '12 at 04:39

Try setting timers (http://nodejs.org/api/timers.html). We do this within our Azure CLI tool to have a spinner that runs when you are doing long running operations.

You can drill into the code here to see how we do it: https://github.com/WindowsAzure/azure-sdk-for-node/blob/master/lib/cli/cli.js. Search for the progress() function.

Node.js - Ensuring a non-blocking call finishes

2 Answers2