0

I'm trying to do a crawler using casperjs. Some requests need raw headers edition: I have to get the raw post data, cookies, etc etc, and once I get them, I'd like to modify them (still raw) and do another request with those modified headers. But I can't find a way to do that.

I've found how to retrieve cookies using Phantomjs, but I did not found anything in casperjs/slimerjs documentation.

Thank you for your help

Alexandre Germain
  • 380
  • 1
  • 16
  • You should be able to retrieve cookies in exactly the same way in slimer.js and CasperJS as in PhantomJS. Are you actually asking about cookies or all request headers? – Artjom B. Sep 21 '16 at 22:02
  • I am asking for the whole request headers, but the only thing I found was how to get cookies – Alexandre Germain Sep 21 '16 at 22:08
  • Does [this](http://stackoverflow.com/a/20931165/1816580) solve your issue? – Artjom B. Sep 22 '16 at 05:24
  • Oh, yeah, from that I was able to filter the URLs to get only pages and not resources, and explicitly override wrong headers. Thank you :) would you like to answer to allow me to mark your post as the accepted answer? – Alexandre Germain Sep 22 '16 at 17:58
  • I'd rather close your question as a duplicate. If your issue is sufficiently different and would have a different answer, then you can answer it yourself (I don't want to do it). By the way, you said that you filtered for page requests. You don't need to do that yourself. CasperJS does that for you if you register to the [`"page.resource.requested"` event](http://docs.casperjs.org/en/latest/events-filters.html#page-resource-requested). You really should use CasperJS-style events than the underlying PhantomJS-style events. – Artjom B. Sep 22 '16 at 19:36

1 Answers1

0

You can listen for the page.resource.requested event and access the headers property of the requestData:

var casper = require('casper').create();
var utils = require('utils');

casper.start('https://example.com/');

casper.on('page.resource.requested', function (requestData, networkRequest) {
  utils.dump(requestData.headers);
});

casper.run();
Grant Miller
  • 27,532
  • 16
  • 147
  • 165