How to get ZombieJS+NodeJS to run over a collection?

Question

My naive approach to get ZombieJS to loop over an huge array of items was to add the following code inside my for loop

var Browser = require("zombie");
var assert = require("assert");

// Load the page from localhost
browser = new Browser()
browser.visit("http://localhost:3000/", function () {

  // Fill email, password and submit form
  browser.
    fill("email", "zombie@underworld.dead").
    fill("password", "eat-the-living").
    pressButton("Sign Me Up!", function() {

      // Form submitted, new page loaded.
      assert.ok(browser.success);
      assert.equal(browser.text("title"), "Welcome To Brains Depot");

    })

});

but all sort of errors ensued. Specially memory errors after trying to load the 100th item.

What am I doing wrong? I wish I could run this every iteration of the loop and also cleaning the resource after every successful try.

You're effectively trying to tell the browser to visit all your items at the same time. Don't call the next `.visit()` until your callback after `pressButton`. — Brad, Jul 14 '14 at 18:47

score 1 · Accepted Answer · answered Jul 14 '14 at 19:04

If it were me, I would create a queue of URLs to visit...

var urls = ['http://example.com/something', 'http://example.com/somethingElse', /* etc. */]

Then, I would create a function that reads from this queue.

function runNextJob() {
    if (!urls.length) {
      return; // W're done!
    }

    var url = urls.pop();
    browser.visit(url, function () {
        // yada yada
        browser.pressButton('Sign Me Up!', function () {
            // On next tick, call runJext Job().  It's important that
            // this is next tick!  Otherwise our call stack gets enormous.
            process.nextTick(runNextJob);
        }
    }
}

To start the cycle, just call runNextJob() once. If you want to run multiple at a time, run that function as many times as you want to have simultaneous jobs. (Say, 4 times for a 4 CPU box, depending on how browser does its work, but you'll want to experiment.)

How to get ZombieJS+NodeJS to run over a collection?

1 Answers1