Iterating over a grid with CasperJS

Question

I am trying to test CasperJS out, and are scraping a site which has a grid layout like:

|Name      |Name      |
|Title     |Title     |
|Image     |Image     |
|Something |Something |
|----------------------
|Name      |Name      |
|Title     |Title     |
|Image     |Image     |
|Something |Something |
|----------------------

If I wasn't using CasperJS I would retrieve a list of all the contains (4 i this case) and then run a method on each container which could retrieve an object with the wanted properties.

I just seem to have a hard time of doing this in CasperJS. First I tried to return the list of DOM elements in casper.evaluate(function(){....}), but it can't return DOM elements.

Then I tried to make an each loop which would push the wanted objects (4) to an array and return it in an Evalue, but it keeps returning null.

How would one go about doing something like this in CasperJS. Can I somehow return a context of a container to a method, which can return the object to the main evaluate, which can the return the collection of the objects?

You're banging your head against the main concept of Casper. The separation between server and client JS. Outside of evaluate, it's server only, no DOM. The bridge is the serializable objects. The two answers explain it well. Notice how the getLinks function in the example returns an array of strings, not DOM nodes.http://docs.casperjs.org/en/latest/quickstart.html — Ruan Mendes, Aug 03 '13 at 08:31

score 9 · Accepted Answer · edited Jul 08 '14 at 16:47

Unfortunately, you can't get a complex structure from evaluate() function, because whatever arg passed from evaluate() is sort of JSON.parse(JSON.stringify(arg)).

But it doesn't mean that you are not able to pass another kind of objects.

Here an example about how get an array with objects from casper.evaluate():

var arrayResult = this.evaluate(function getGridResuls(){

    //create array
    var arrayObjects = new Array();

    //Iterates over table (grid) elements
    jQuery("table.results").each(function( index ) {

        //get table (grid)
        var tableResult = jQuery(this);

        //create basic object    
        objResult = new Object();

        //fill object properties
        objResult.name      = tableResult.find('selector to get name').text();
        objResult.title     = tableResult.find('selector to get title').text();
        objResult.image     = tableResult.find('selector to get image info').text();
        objResult.something = tableResult.find('selectot to get something').text().trim();

        //assign object to array
        arrayObjects[index] = objResult;

    });  

    //return array with objects
    return arrayObjects;

});

...
//do something with arrayResult

I'm assuming that the web context includes the JQuery library.

Tip: try to run the js code of the evaluate() function by using the browser console in order to be sure that your js code is working as expected.

score 2 · Answer 2 · answered Jul 25 '13 at 07:55

2

The approach is correct but evaluate is sandboxed. In addition, the arguments and the return value to the evaluate function must be a simple primitive object but if it can be serialized via JSON, then it is fine. Closures, functions, DOM nodes, etc. will not work!

Instead of returning wanted object, returns a serialized version of wanted object using JSON.stringify()

answered Jul 25 '13 at 07:55

Cybermaxs

24,378
8
83
112

Thanks for your reply, but how would you then DRY it up. I would to have one method which can take a DOM context and return a JSON object (stringified). Won't I be able to call functions within the sandbox and in that return the DOM element? Basically I just want to find out what the best solution is to iterate over a container, and get the elements out of that container in CasperJS. – Dofs Jul 25 '13 at 08:19
@dofs there's no DOM outside of evaluate, you just can't do it. You have to abstract your elements into simple serializable objects – Ruan Mendes Aug 03 '13 at 08:26

Iterating over a grid with CasperJS

2 Answers2

Linked