1

I'm trying to run a simple script that will scrape some data using x-ray and insert it into my Events collection.

    if (Meteor.isServer) {
      var Xray = Meteor.npmRequire('x-ray');
      var xray = new Xray({
        version: "2.0.3"
      });

      xray('http://www.events12.com/seattle/january/', '.qq', [{
        title: '.title',
        date: '.date',
        link: 'a @href',
        allContent: '@html'
      }])(function(err, content) {

        for (var i = 0; i < content.length; i++) {
          (function() {

            console.log(i);

            var newEvent = {
              owner: 'me',
              name: content[i].title,
              date: content[i].date,
              url: content[i].link,
              createdAt: new Date(),
              description: 'none'
            };

            console.log(newEvent);

            Events.insert(newEvent, function(err, data) {
                console.log(err);
                console.log(data);
            });


          })();
        }
      });
    }

The callback from x-ray that takes in content has all the scraped data in an array of objects, each with several properties. When I try to insert this data into my Events collection, the for loop iterates once and then exits, but no error is shown. If I remove the Events.insert() the loop iterates all the way through.

What am I missing? What is the proper way to execute such a task?

S. Heutmaker
  • 146
  • 1
  • 6
  • 1
    I'd expect to see an error along the lines of "Meteor code must be in a fiber, try using Meteor.bindEnvironment" in the server console. – user3374348 Jan 22 '16 at 07:39
  • 1
    What does your events schema look like? My guess is that the anonymous function is hiding an exception in your insert call. – Stephen Woods Jan 22 '16 at 13:59
  • 1
    I was going to try to repro the problem, but wanted to avoid adding the packages so I just made content = fake data. It worked just fine. Is the x-ray callback non-blocking? Maybe there's a problem trying to insert before all the data is available? – terrafirma9 Jan 22 '16 at 15:32

1 Answers1

0

The Events.insert() was being called outside of any Meteor fibers. Adding Meteor.bindEnvironment() and feeding the entire function in as a callback fixed this problem.

S. Heutmaker
  • 146
  • 1
  • 6