1

I have an weird error and can't find the cause of it for the last few hours...

I have a meteor app, that scrapes some webpages for information and everything works fine as long as I use reuqest and cheerio for static pages, but now I have a dynamic site and I wanted to use phantomjs, casperjs and spookyjs for this one, but here I get some bug... My code is as follows, I import the npm modules at the start:

    if (Meteor.isServer) {
    var cheerio = Meteor.npmRequire('cheerio');
    var request = Meteor.npmRequire('request');
    var phantomJS = Meteor.npmRequire('phantomjs');
    var spooky = Meteor.npmRequire('spooky');

And sometime later I want to use spooky to scrape some webpage:

 spooky.start("https://www.coursera.org/");

  spooky.then( function () {
    this.fill("form", {email: user, password: pass}, true);
  });`

But as soon as I call the method I get the following error message:

    20150224-21:16:39.100(-5)? Exception while invoking method 'getLecturesCoursera' TypeError: Object function Spooky(options, callback) {
    ....
    I20150224-21:16:39.281(-5)? } has no method 'start'
    I20150224-21:16:39.281(-5)?     at [object         Object].Meteor.methods.getLecturesCoursera (app/moocis.js:72:14)

I am doing something completly wrong and I have no clue why it isn't working... I tried to verify that spookyjs and phantomjs are installed correctly in my app, but that isn't as easy as it sounds for someone who uses them for the first time...

Carnivean
  • 51
  • 5

1 Answers1

2

Like normally with spooky you have to create a new Spooky Object before you can start and run it...

 if (Meteor.isServer) {
  Meteor.startup( function () {
    var Spooky = Meteor.npmRequire('spooky');

    var spooky = new Spooky({
          child: {
              transport: 'http'
          },
          casper: {
              logLevel: 'debug',
              verbose: true,
              ignoreSslErrors: true,
              sslProtocol: 'tlsv1'
          }
      }, function (err) {
          if (err) {
              e = new Error('Failed to initialize SpookyJS');
              e.details = err;
              throw e;
          }

          spooky.start(
              'https://www.google.com');
          spooky.then(function () {
              this.emit('hello', 'Hello, from ' + this.evaluate(function () {
                  return document.title;
              }));
          });
          spooky.run();
      });

      spooky.on('error', function (e, stack) {
        console.error(e);

        if (stack) {
          console.log(stack);
        }
      });

      spooky.on('hello', function (greeting) {
          console.log(greeting);
      });

      spooky.on('log', function (log) {
          if (log.space === 'remote') {
              console.log(log.message.replace(/ \- .*/, ''));
          }
      });
  })
}
Carnivean
  • 51
  • 5