0

I'm building a scraper with Scraper Wiki, here: https://scraperwiki.com/scrapers/fashfinder/edit/#

Without boring you with too many details, I load up about 120 links into an array, $allLinks. Then, at the bottom of the page, I call a FOR loop on the array as follows:

for ($i = 0; $i<count($allLinks); $i++){
   getInfo($allLinks[$i]);
};

getInfo() is a function that gets product information from the supplied links. The function works fine for $i = 0 and $i = 1. Then, when $i = 2 something breaks - the source is loaded in the scraper and the whole thing stops. No errors but it says "Exit Status 139".

I tried reversing the $allLinks array with $allLinks = array_reverse($allLinks); but the exact same thing occurred - it scraped 2 pages, got to the 3rd then suddenly stopped.

Any ideas on what's going wrong here? Can't find much info on what "EXIT STATUS 139" means, especially regarding scraperwiki!

JVG
  • 20,198
  • 47
  • 132
  • 210
  • 1
    Do a barrell roll xD....put some info on the var_dump or print_r on the $allLinks var please – Hackerman Feb 23 '13 at 04:41
  • @RobertRozas Here ya go http://pastebin.com/LNMtds2Z pretty much the correct output - it's grabbing 122 links. The problem is with the FOR loop but no idea what. – JVG Feb 23 '13 at 04:44
  • @RobertRozas Even when I set ` for ($i = 0; $i<100; $i++){` i get the error, looks like a problem on scraperwiki's end? – JVG Feb 23 '13 at 04:46
  • 2
    @RobertRozas The issue is most likely in the getInfo function. Comment it out and just put a `echo 'test';` or something to see it work. – sachleen Feb 23 '13 at 04:53
  • @sachleen has a good idea. – Nick Pickering Feb 23 '13 at 04:57
  • I think the same based in the pastebin...you should write your getInfo function. – Hackerman Feb 23 '13 at 04:58
  • and I don´t recommend you use count($allLinks) as your validator... it will count the entire array each time. It would be better to save the count on a var before the for... or use the foreach – Manatax Feb 23 '13 at 05:02
  • @RobertRozas The getInfo function is in the original link in the OP. Here's a pastebin of the function on its own though http://pastebin.com/XGyVdmAu – JVG Feb 24 '13 at 12:48

1 Answers1

0

May be trying this:

foreach ($arrayLinks as $key => $value) {
  getInfo($value);
}

PS: Your var_dump of the array looks fine, maybe ut's a issue with your getInfo function.

Saludos ;)

Hackerman
  • 12,139
  • 2
  • 34
  • 45
  • the foreach will not solve the problem, but I agree it will be more efficient than the current for. – Manatax Feb 23 '13 at 05:04
  • Thats right....but like i said....i think that the problem may exist in the getInfo() function. – Hackerman Feb 23 '13 at 05:06
  • @RobertRozas Here is the getInfo() function: http://pastebin.com/XGyVdmAu And here is the code in its entirety: http://pastebin.com/SZ8XFCaF – JVG Feb 24 '13 at 12:50
  • I recently found this information: exit code 139 is signal 139-128=11, segmentation fault. This could be a null pointer dereference or heap corruption. It's undeniably a bug, but the cause could be pretty much anything. – Hackerman Feb 25 '13 at 12:31