how can casperjs click() tag not an anchor

Question

target html is like

<script src="http://static.com/js/qiyi/config.js" type="text/javascript">
<dl class="selected">
   <dt>
       <span data-value="2014">2014年</span>
   </dt>
</dl>
<dl>
   <dt>
      <span data-value="2013">2013年</span>
   </dt>
</dl>

is not a simple anchor tag, so in my browser, when i click the span tag, it will call the config.js to do something(i dont know what does it did), it will get change the content. but when i use

this.click()

to simulate the mouse action, it doesnot work. because i know the casperjs can work like a browser. if i want to crawl different years content, how can i do

my code is

var casper = require("casper").create({
    clientScripts: [
        "jquery-1.7.2.js",
        "config.js"],  //in url page call this js
    remoteScripts: ["http://static.qiyi.com/js/qiyi/config.js"],//also the config.js

    stepTimeout: 120 * 1000,  //单步超时时间
    pageSettings: {  
        loadImages: false  
    },  
    verbose: true,  
    logLevel: "error"  
});   

var fs = require('fs');
var filename = 'content.txt';

casper.on('resource.received', function(resource) {
    casper.echo(resource.url);
});

casper.on('click', function(resource) {
    casper.echo("click");
});

casper.on("page.error", function(msg, trace) {
    this.echo("Error: " + msg, "ERROR");
});

casper.on("resource.error", function(resourceError {
    console.log(JSON.stringify(resourceError));});

casper.start("http://www......");
casper.then(function() {
    this.evaluate(function(){
        document.querySelectorAll('[data-value="2013"]')[0].click();
        //it does not work
    });
    this.echo("click");
});
casper.then(function() {
    this.click('span[data-value="2013"]');
    //it does not work
    fs.write(filename, this.getHTML(), 'w');
    this.echo("click");
});

casper.run();

`this` is probably a reference to the Casper object. Where are you calling it from? — bob_cobb, Aug 13 '14 at 07:11
i have edit the question. it does not work. so i suspect the config.js can not in the html by casper — liubochao988, Aug 13 '14 at 07:42
Please register to the [`remote.message`](http://docs.casperjs.org/en/latest/events-filters.html#remote-message) and [`page.error`](http://docs.casperjs.org/en/latest/events-filters.html#page-error) events. Maybe there are errors. If bind is an issue, you need a [shim](https://github.com/ariya/phantomjs/issues/10522#issuecomment-39248521). — Artjom B., Aug 13 '14 at 08:54
Please register to the [`resource.error`](http://docs.casperjs.org/en/latest/events-filters.html#resource-error) to see it is actually the resource error like this: `casper.on("resource.error", function(resourceError){console.log(JSON.stringify(resourceError));});` — Artjom B., Aug 13 '14 at 21:41
@ArtjomB. thank you for you help, i have update my code, but it still not work, my friend who is familiar with UED, tell me this config.js like a contextual, so i can not simulate that click function, i have learn some knowledge to use crawl, in this case, i have nothing to do, thanks a lot for you help, and i also hope this problem can be solved. — liubochao988, Aug 14 '14 at 02:07
@ArtjomB. i have update the code, and add some event, in log i found the click has work, and received http://cache.video.qiyi.com/sdvlst/6/1300000224/2013/?cb=scDtVdListC , but i use this.getHTML(), and save it, i still 2014, not 2013 — liubochao988, Aug 14 '14 at 02:40
Your question doesn't make sense. If this is a production page and the config.js is included in the page, why do you try to include it through casper again? You probably can't overwrite configurations that are done in the original config.js with your own, but that depends on the actual code of config.js. Please some string to distinguish between the events. Also, have you tried waiting a little after the click? `this.wait(5000, function(){fs.write(...);});` — Artjom B., Aug 14 '14 at 07:41
@ArtjomB. thanks a lot for you help. it is worked. because i am a newbie, i am not insure about the page if include in. at present, the casperjs can work like browser, and include all the js environment. — liubochao988, Aug 15 '14 at 03:06
possible duplicate of [PhantomJS; click an element](http://stackoverflow.com/questions/15739263/phantomjs-click-an-element) — , Aug 15 '14 at 03:16
@ArtjomB. i have a little confuse, how can i know the exact time that i must wait to get the resource, because you write 5000. is it have any api to know the resource have all get in? — liubochao988, Aug 15 '14 at 03:17
If the data is loaded as an ajax request, you can specifically wait for this request to finish if you know something about the url. For example: `this.waitForResource(/json/, function(){...});` if it has `json` somewhere inside it by using a regular expression. — Artjom B., Aug 15 '14 at 08:20

score 1 · Accepted Answer · answered Aug 15 '14 at 03:14

if you first use js, you must be sure about how to debug,include page.error and capture event is important.like

casper.on("click", function(){this.echo();});
casper.on("page.error", function(){this.echo();});

the event happen you can get. for this problem, after click, must have some time to receive the resource.

this.wait(5000, function(){fs.write(...);})

thank you @ArtjomB.

score 0 · Answer 2 · answered Aug 13 '14 at 07:17

0

You probably want to evaluate some javascript (that has access to the window object as if you were in the browser) with casper.evaluate

casper.evaluate(function() {
  document.querySelectorAll('[data-value="2014"]')[0].click();
});

answered Aug 13 '14 at 07:17

bob_cobb

2,229
11
49
109

i have try this, it does not work. and i dont know the casper how to work, like – liubochao988 Aug 13 '14 at 07:33
Is the an actual link? – bob_cobb Aug 13 '14 at 07:51
i do not know about that, when i click , it call the config.js, and the content of html change – liubochao988 Aug 13 '14 at 08:00

score 0 · Answer 3 · answered Aug 13 '14 at 07:55

0

Use the clickLabel function, outside of the evaluate function Works for span's in my code

casper.then(function(){
    this.clickLabel('2014', 'span');
});

Found here

answered Aug 13 '14 at 07:55

Ka0s

398
4
10

thank you for you help, but it doesnot work, maybe the span dont have a href? – liubochao988 Aug 13 '14 at 08:13
My spans all have hrefs in them which could be the problem you have then – Ka0s Aug 13 '14 at 08:15
its mean i have to know how does the config.js work? and simulate? but the config.js have compressed, so i can not know that – liubochao988 Aug 13 '14 at 08:19
I dont know if this will help much, but online tools exist like [JS NICE](http://www.jsnice.org) to decompress js. Not sure if that will help – Ka0s Aug 13 '14 at 08:24
@liubochao988 This cannot work because the text must match perfectly: `clickLabel('2014年', 'span')`. – Artjom B. Aug 13 '14 at 08:52
@ArtjomB. thank you for help.when i use that, it get error, so i have change 2014年. but i still doesnot work. so i thought the casper cannot load config.js – liubochao988 Aug 13 '14 at 08:57
Look at adding client scripts [here](http://casperjs.readthedocs.org/en/latest/modules/casper.html#index-1) – Ka0s Aug 13 '14 at 09:02

how can casperjs click() tag not an anchor

3 Answers3