Casper JS Logging and Downloading Once Submit Form

Question

I am trying to traverse a state database of court cases for homicides and other violent crimes research, using Casper.JS

The database works by the client first agreeing to Terms of Service via submitting an HTML form. After that a second form is presented in which you can enter in search terms to find up to 500 records in an HTML table.

This state database/site is very unreliable and often breaks down. The goal is to traverse it with Casper and host the data independently for a client.

My problem is that I can't get basic Casper tasks to work, besides loading the first form. It appears to be navigating, hits an about:blank and then stops. It doesn't log at critical junctures even though it is coded to do so.

The script I am running:

var casper = require('casper').create({
  clientScripts: [
    'jquery.min.js', // These two scripts will be injected in remote
    'moment.min.js' // DOM on every request
  ],
  pageSettings: {
    loadImages: false, // The WebPage instance used by Casper will
    loadPlugins: false // use these settings
  },
  logLevel: "debug",
  verbose: true // log messages will be printed out to the console
});
casper.start('http://casesearch.courts.state.md.us/casesearch/processDisclaimer.jis', function(status) {
  var y = (this)
  $("input[name='disclaimer']").prop('checked', true);
  $("input[name='action']").click();
  casper.log(y, 'info');
  setTimeout(function() {
    y.download('http://casesearch.courts.state.md.us/casesearch/inquiry-results.jsp?middleName=&partyType=&lastName=T&filingEnd=&filingDate=&site=00&filingStart=&d-16544-p=20&countyName=&action=Search&courtSystem=B&firstName=N&company=N', 'phantom.csv')
  }, 3000);
});

casper.run();

The output I am getting:

[info] [phantom] Starting...
[info] [phantom] Running suite: 2 steps
[debug] [phantom] opening url: http://casesearch.courts.state.md.us/casesearch/processDisclaimer.jis, HTTP GET
[debug] [phantom] Navigation requested: url=http://casesearch.courts.state.md.us/casesearch/processDisclaimer.jis, type=Other, willNavigate=true, isMainFrame=true
[debug] [phantom] url changed to "http://casesearch.courts.state.md.us/casesearch/processDisclaimer.jis"
[debug] [phantom] Automatically injected jquery.min.js client side
[debug] [phantom] Automatically injected moment.min.js client side
[debug] [phantom] Successfully injected Casper client-side utilities
[info] [phantom] Step anonymous 2/2 http://casesearch.courts.state.md.us/casesearch/processDisclaimer.jis (HTTP 200)
[debug] [phantom] Navigation requested: url=about:blank, type=Other, willNavigate=true, isMainFrame=true
[debug] [phantom] url changed to "about:blank"

Can someone instruct me as to why I'm not getting my console.log(y) and why I have no download or error message? It just terminates once that output is given.

I should also mention that I believe the download will only work if the database server knows that I accepted the Terms, as in, I need to send my cookies to it in my download request. How does Casper handle that?

Thank you.

You should know: CasperJS [can't download more than 7 MB file](https://github.com/casperjs/casperjs/issues/362), you must try with smallest-size files Also read [this issue](http://stackoverflow.com/questions/40342285/unable-to-download-zip-file-from-url-with-casperjs) — , Jan 08 '17 at 16:51
@Igor I did not realize that was a limitation, that's too bad. :( — Summer Developer, Jan 09 '17 at 01:49

Casper JS Logging and Downloading Once Submit Form

0 Answers0