3

I have a perl script that navigates through two form pages on a website in order to download the result of final form submission to a file. This page is very large and I'd rather write it directly to disk instead of having the whole thing sitting in memory.

Here's a code snippet:

$mech->submit_form(
    form_name => 'search',
    fields      => {
        "srch_recd"    => $cfg{max_rows}, #results per page
    }
);

$mech->save_content( $workdir.$cfg{cachedstones} );

I know that I can do this:

$mech->get( $url, ":content_file"=>$tempfile );

in order to have the result go to a file when using get(). However, doing the same hasn't worked for submit_form().

How can I get the result of a submit_form() to go into a fiel directly, rather than sit in memory?

Jessa
  • 1,549
  • 2
  • 12
  • 26
  • http://stackoverflow.com/questions/2263662/how-can-i-download-a-file-using-wwwmechanize-or-any-perl-module or http://stackoverflow.com/questions/1727923/how-do-i-download-a-file-with-wwwmechanize-after-it-submits-a-form – mpapec May 31 '13 at 18:43
  • I looked at those questions, and they're not quite addressing the same problem. The code I already have works. I submit the form and get a results page that I save with`$mech->save_content( $workdir.$cfg{cachedstones} )`. However, this page is very large and it's wasting an unacceptable amount of memory. I want to optimize my script so that the file is written as the results page is coming in. – Jessa May 31 '13 at 18:57

1 Answers1

0

From WWW::Mechanize::FAQ

You can also save any content directly to disk using the :content_file flag to get(), which is part of LWP::UserAgent.

$mech->get( 'http://www.cpan.org/src/stable.tar.gz',
            ':content_file' => 'stable.tar.gz' );

That should work with ->post, too.

Andy Lester
  • 91,102
  • 13
  • 100
  • 152
  • You can also construct a request object with HTTP::Request::Common POST(), and then call `$mech->request($req_object, $filename)` – runrig May 31 '13 at 21:46
  • 1
    Or (looking up what I've done before) set the current_form, construct a request object with `my $req = $mech->current_form()->click()` and then `$mech->request($req, $filename)` – runrig May 31 '13 at 22:45