Server-sided issues when scraping with Node JS Cheerio module?

Question

I am trying to follow this thread here: How can one parse HTML server-side with Meteor?

Unfortunately I get the following errors when doing so:

Uncaught Error: Can't make a blocking HTTP call from the client; callback required.

Here is the javascript code for my project:

var cheerio;

if (Meteor.isClient) {

  Template.entry.events = {
    'click .btn_scrape' : function() {
    $ = cheerio.load(Meteor.http.get("https://github.com/meteor/meteor").content);
    console.log($('.commit-title').text().trim());
    },
 }
}

if (Meteor.isServer) {
  Meteor.startup(function () {
    var require = __meteor_bootstrap__.require;
    cheerio = __meteor_bootstrap__.require('cheerio');
  });


}

if I put the code in Meteor.startup(function()... nothing happens, there is no error and nothing is logged to the console.

I'd like to be able to call a function when a button is clicked to get the content in a textbox and scrape it, but this I can do later once I get the code working.

Would anyone by chance know how to fix this?

Thank you for your time,

Jonathan.

score 3 · Accepted Answer · edited May 23 '17 at 12:07

The server and client side are still segregated. In that other post Meteor.call is used to relay the message to the server to do a request there and return the scrape result back to the client.

The error you're getting is due to javascript being asynchronous on the browser side of things. More info about that here & here. You need to use a callback with client side code because it takes time to fetch data from the server.

Is this your intention to run the http request off the client? On the client there are issues such as the Access-Control-Allow-Origin.. Which is why on that post a Meteor.call is done to the server to proxy the request through and return the data to the client.

In your click handler you could use the code at How can one parse HTML server-side with Meteor? with:

Template.entry.events = {
 'click .btn_scrape' : function() {
    $('.btn_scrape').attr('disabled','disabled')
    Meteor.call("last_action",function(err,result){
        $('.btn_scrape').removeAttr('disabled')
        console.log(result);
    });
 }
}

In the Meteor.isServer section of your code you would still need the method last_action to proxy the data to your browser:

var cheerio = __meteor_bootstrap__.require('cheerio');
Meteor.methods({
last_action: function() {
       $ = cheerio.load(Meteor.http.get("https://github.com/meteor/meteor").content);
       return $('.commit-title').text().trim()      
    }
})

Server-sided issues when scraping with Node JS Cheerio module?

1 Answers1