1

For a project I have to make a web crawler. Seems to working fine up to a certain extent. The code is:

var Crawler = require("crawler");
var validUrl = require('valid-url');

var c = new Crawler({
  maxConnections : 10,
  callback : function (error, result, done) {
    if(error){
      console.log(error);
    }else{

      var $ = result.$;
      var page = result.body;
      var res = page.match(/pizza/i);

      if(res && res.length > 0){
        console.log($("title").text())
      }

      $("a").each(function(index,a){
        if (validUrl.isUri(a.attribs.href)){
          c.queue(a.attribs.href);
        }
      })
    }
    done();
  }
});

c.queue('https://en.wikipedia.org/wiki/Pizza');

The error I'm getting is:

 $("a").each(function(index,a){
        ^

 TypeError: $ is not a function

Doesn't make sense because running it in console, I print out a decent list using the $ already and eventually it says not a function. Anyone know how to fix this?

CertainPerformance
  • 356,069
  • 52
  • 309
  • 320
  • `I print out a decent list using the $ already` Do you mean that the `console.log($("title").text())` line does indeed print out something? (if not, sounds like the `result` object simply doesn't have a `$` property) – CertainPerformance Dec 07 '18 at 05:46
  • https://stackoverflow.com/questions/3931529/is-not-a-function-jquery-error – Ramesh Kumar Dec 07 '18 at 05:46
  • Is this the full code? – Janith Dec 07 '18 at 05:54
  • Does this script execute prior to the jQuery library being included on the page? – m4rk Dec 07 '18 at 06:12
  • Title's are being printed out. And links are being sent in correctly. – Hammad Ahmad Dec 07 '18 at 06:17
  • Yes this is the full code, not sure what you mean @m4rk – Hammad Ahmad Dec 07 '18 at 06:18
  • @m4rk yea i realize what you're saying, it does. it executes a bunch of times and then eventually gives that error. i think it has to do with noConflict but I'm not just how to incorporate that into NodeJS – Hammad Ahmad Dec 07 '18 at 07:45
  • Okay. See if this works - in the script above, replace '$' with 'jQuery'. Do you get a 'jQuery is not a function' error? – m4rk Dec 07 '18 at 07:53
  • Your description is not full of error! `warn: CRAWLER response body is not HTML, skip injecting. Set jQuery to false to suppress this message` – hoangdv Dec 07 '18 at 08:40
  • Just check type of $ before using it. `if(typeof $ !== 'function') return` – hoangdv Dec 07 '18 at 08:42
  • @hoangdv ok so the code runs now fine, but I still get the [warn: CRAWLER response body is not HTML, skip injecting. Set jQuery to false to suppress this message] and also uRI protocol [error: CRAWLER Error Error: Invalid protocol] which comes with many links, I have no idea how to fix these 2, not breaking errors but still want to see if i can get rid of them – Hammad Ahmad Dec 07 '18 at 19:11

0 Answers0