1

I am having trouble parsing a page with CasperJS. It seems like the this.evaluate() function only handles the source code. However, the page I am looking at uses jQuery to add classes to specific elements after the document has been fully loaded.

How can I use CasperJS to evaluate the generated source after page is fully loaded / ready.

casper.start(url);

casper.userAgent('Mozilla/5.0 (Windows NT 6.1; WOW64) Chrome/30.0.1599.101');
casper.viewport(1920, 20000);

casper.on('load.finished', function(resource) {
    this.evaluate(getSelectedItems);
});

casper.run();

.

This is the code that I am currently trying to use within my this.evaluate(getSelectedItems) function. I just want to detect how many selected items there are, but the source code doesn't contain .selected-item. That class is generated after the full page load with jQuery.

function getSelectedItems() {
    return document.querySelector('.selected-item').length;
}
bbullis
  • 532
  • 7
  • 20
  • that implies u would return an array of items, u cant call innerHTML on an array – j_mcnally Jan 10 '14 at 23:08
  • in my first example, even with multiple items on the page, it would of selected first valid item it found in the code and returned it's innerHTML. I updated question to better suit my needs. Thanks – bbullis Jan 10 '14 at 23:13
  • Is `getSelectedItems` in the client scope or the PhantomJS scope? See http://stackoverflow.com/questions/12222856/passing-arguments-to-anonymous-function-inside-page-includejs-and-page-evaluat/12223183#12223183 – nrabinowitz Jan 10 '14 at 23:35
  • @nrabinowitz - not exactly sure what you mean. I know CasperJS utilizes PhantomJS and because I am calling this.evaluate, I take it CasperJS is calling over to PhantomJS. However, I am not injecting jQuery into the page, jQuery already lives on the page I am trying to parse. – bbullis Jan 10 '14 at 23:45
  • ...sorry, thought that `getSelectedItems` might not have been in scope. But that's probably not the case here. – nrabinowitz Jan 11 '14 at 00:04

1 Answers1

0

If the elements are not available right away, wait for them:

casper.waitForSelector('.selected-item', function() {
    this.echo(this.evaluate(getSelectedItems));
});
nrabinowitz
  • 55,314
  • 10
  • 149
  • 165
  • Thank but, I tried that and it ends up timing out. Have you been able to verify that in your script that it works? – bbullis Jan 11 '14 at 01:02
  • I can't verify without knowing what you're scraping, but I use CasperJS to look for classes added asynchronously all the time, using this or similar `waitFor` methods. Likely suspects if this doesn't work are a bad scraping function or a bad selector... – nrabinowitz Jan 11 '14 at 06:23