0

I'm working with WWW::Mechanize to slurp a products catalog from a web site into our database (Ingram Micro). Everything is over SSL.

I'm receiving a random error like the following:

Protocol scheme 'https' is not supported (LWP::Protocol::https not installed)

...but, LWP::Protocol::https is installed. In fact, everything works fine most of the time. The only thing I can think of is that this has something to do with using threads on Windows (the process splits the job to 25 threads to compensate for the long time Ingram's website take to deliver each page). I haven't seen the error (so far) when I use a single thread.

The error doesn't happen everytime and generally only happens for one thread, the rest can work without receiving it.

However, this is really wierd. I'd like to know if anyone here has seen something like this before or if someone has any idea of why this might happen.

Thanks,

Francisco

Edit: Just in case someone wonders, I'm on Windows 7 x64 and Perl 5.16.3 x64 built with MSVC10.

Francisco Zarabozo
  • 3,676
  • 2
  • 28
  • 54

2 Answers2

1

It is likely a problem with a module not being thread safe. See this Perlmonks discussion, also about LWP and https.

The thread (er...discussion) also offers some potential solutions.

0

The solution I use is cloning Mechanize object at the start of each thread and working with the cloned version. But as I said I'm using WWW::Mechanize not plain LWP.

$mech = $mech->clone();
gangabass
  • 10,607
  • 2
  • 23
  • 35
  • I'm actually creating a new instance of WWW::Mechanize on each thread right after they are created. I guess that should work just as well as clone (or better), if that was the issue. I'll try it anyway. – Francisco Zarabozo Apr 04 '13 at 17:10