-1

I need to fetch a list items available in a website that list the items in paging style. I will extract information of the items on each pages. I wrote a Javascript code using a settimeout(function(){...code...}, 5000), and run this code in the Firefox web console. The script fetches all the items in the page, once this is done, the script submits the page by executing click() method of the hyperlink "next". this will continue till the page has "Next" hyperlink in the page (in last page only "back" hyperlink if present). On every page submit I timeout it to 5 seconds.

I executed this code in the Firefox web console, this run successfully and fetches the first pages results, then submits the page, the next page too loads. But, the problem is, once the page is submitted the script stops. How to run this script continuously even after the next page load? Is there any other way to achieve this functionality?

var inc=0;
var ids;
var main=function(){
console.log("attempiting..............."+inc);
var i=0;
{
    ids[inc]=b();//function, fetch the items from the page
    console.log(ids);
    var more=document.getElementById("m_more_item");
    if(more){
        //next link is there
        var cli=more.getElementsByTagName("a");
        console.log("navigating to next page "+cli[0]);
        inc++;
        setTimeout(main,5000); //I tried placing this after click, but didnt work!
        cli[0].click(); //submitting...
    } else {
        //last page reached, exiting...
        console.log("exit1-> total items fetched are: "+ids);
        return ids;
    }
}
};
main();
Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
vvra
  • 2,832
  • 5
  • 38
  • 82
  • Javascript is a client-side language that is embedded in your HTML page. When you change to another page, the running script is cancelled. Would be a bit silly to have a js script running of a previous website. You could use query parameters, handle those with a server-side language and use those parameters to simulate a resume of your script. – Tim S. Nov 01 '12 at 09:27
  • 1
    If you can explain what you are trying to do more clearly we can help it. But as I understand in your case you can use a sync AJAX call. – Onur Topal Nov 01 '12 at 09:34
  • All I am trying to do is fetch a list of items from a webpage for the given id, manually doing this and aggregating them takes very long time. the webpage does not list the items in a single fetch, but in paging. I have a set of ids[], I call the webpage url with every ids[i] as request param, the response (in html) will be 1st 10 items (for this ids[i]), parse, iterate them, prepare the data, next, call the same url with the additional param for paging, &start=10 and so on till the last page.(I identify last-page by missing "next" link). I repeat this process for all the ids in my list. – vvra Nov 01 '12 at 10:29
  • Yes I agree, I shall recode this with ajax. get the response html in a div or IFRAME to parse. I welcome any other easy & better ideas. :-) – vvra Nov 01 '12 at 10:30

1 Answers1

0

setTimeout call make you function recursive and keep looping this should work.

setTimeout(cli[0].click, 5000);

Cheers.

EDIT: It was long for a comment so I am adding it as a answer.

You can use AJAX calls in separated page first call the page with AJAX with start=1 and on success of this request parse the data and collect information as you need and do not forget the keep lastItemIndex for your next request and post the information you need with another AJAX call on success of the second AJAX call first one with start=lastItemIndex+1.

These calls should do what you want.

I suggest using jQuery for this as it will be easy to call AJAX and parse the response.

Onur Topal
  • 3,042
  • 1
  • 24
  • 41