HTTP 407 CasperJS through corporate proxy

Question

I'm trying to use casperjs to access creditkarma.com (which isn't blocked by the firewall). But I keep getting a 407 error and a page that says I need to provide a username and password, no form just an error. I tried http://username:password@example.com. I also tried --proxy with --proxy-auth I also tried sending headers.

casper.options.pageSettings = {
    customHeaders:{
        'Authorization':'Basic '+btoa('myUserName:myPassword')
    }
}

The command is running through the windows command prompt so I also set HTTP_PROXY. I have spent a few hours on this and cannot seem to figure out why this won't work. Does anyone have other ideas I can try?

Please, show how you launch script. Also, corporate vpn or corporate proxy? — Vaviloff, Nov 27 '15 at 08:06
Casperjs test ck.js inside Casper is a call to phantomjs--ssl-protocol=any and then after that I included the proxy options. I'm not exactly sure. There is a Vpn I connect to, but I have a proxy url that is mentioned in our internal forums — Michael St Clair, Nov 27 '15 at 08:09
I just looked at the header sent and it is actually proxy-authorization negotiate. Is there a way to encode for that? — Michael St Clair, Nov 27 '15 at 08:25

score 2 · Accepted Answer · edited May 23 '17 at 12:07

2

Judging by 407 response you have to 1. Use proxy 2. Supply proxy auth requisites.

In CasperJS (and PhantomJS) you do:

casperjs --proxy=192.168.0.100:8080 --proxy-auth=user:pass ck.js

Also, seeing as the target site uses https protocol only, you would do well to include in the list of arguments instructions to prevent ssl errors.

casperjs --ignore-ssl-errors=yes --ssl-protocol=any --proxy=192.168.0.100:8080 --proxy-auth=user:pass ck.js

In case NTLM authorization required, it is present in PhantomJS from version 2.0.0, but CasperJS doesn't yet support that version. You could enable a local NTLM proxy to bypass that restriction and still use CasperJS as shown in this answer: cntlm proxy with phantomjs

edited May 23 '17 at 12:07

Community

1
1

answered Nov 27 '15 at 08:18

Vaviloff

16,282
6
48
56

I tried that and it doesn't work. I did notice that the headers are negotiate and not basic, do I need to do something different for that – Michael St Clair Nov 27 '15 at 08:28
I'm not fluent in windows networking, could that be NTLM authentication? It seems that PhantomJS supports it [from version 2](https://github.com/ariya/phantomjs/issues/11037#issuecomment-15079892) bur CasperJS does not support v2. There's a good workaround with an another tool: http://stackoverflow.com/questions/31209449/cntlm-proxy-with-phantomjs – Vaviloff Nov 27 '15 at 08:50
I ended up using the cntlm local proxy, however its not a great solution because you need admin rights to use it which means my script is not easy to run for other users. And as you noted casperjs does not work with the version of phantomjs that works with ntlm – Michael St Clair Dec 03 '15 at 20:41

HTTP 407 CasperJS through corporate proxy

1 Answers1