2

I have a large test suite for Catalyst/PSGI website that takes about 40+ minutes run. I've switched it to use aggregated tests, using Test::Aggregate::Nested, and now it takes about 20+ minutes to run.

I've been experimenting with having the test suite use the clone method from WWW::Mechanize between tests to improve performance.

Between tests, I am cloning and "resetting" the shared $mech object like so:

if ($orig) {

    $mech = $orig->clone();
    $mech->max_redirect(7);
    $mech->cookie_jar->clear;

} else {

    $orig = Test::WWW::Mechanize::PSGI->new( app => $app );

}

Note that Test::WWW::Mechanize::PSGI extends WWW::Mechanize.

This reduces the time it takes to run tests significantly, to under 5 minutes.

But clearly this isn't enough: I still need to get a new object for specific tests, because of side-effects between tests that requires manually getting a new Test::WWW::Mechanize::PSGI object in some of the test scripts. That adds another few minutes to the test time.

I consider the side-effects between tests a bug and the neet to get a fresh object in some tests to be a kluge.

So my question is: what else can I do to reset the state of the object?

simbabque
  • 53,749
  • 8
  • 73
  • 136
Rob
  • 781
  • 5
  • 19
  • 1
    There was a talk at YAPC::EU 2014 by Tatsuro Hisamori about using a distributed system to run a large test suite. This sounds like an example where that might make sense. I couldn't find the video, but here are the slides: http://de.slideshare.net/myfinder/yapceurope2014myfinder – simbabque Oct 09 '14 at 08:33
  • I'm not looking to run tests in parallel. I'm looking for information on what the side-effects in the cloned Test::WWW::Mechanize::PSGI object are. – Rob Oct 09 '14 at 09:15

1 Answers1

0

It looks like the clone method aore WWW::Mechanize does not produce a pristine clone, despite the documentation. I needed to add the following, which appears to be fixing the issue for most tests:

$mech->{headers} = {};
$mech->{page_stack} = [];

FWIW, I found these by using an is_deeply test to compare the clone with a new object.

Rob
  • 781
  • 5
  • 19