I have been trying to use Octoparse to scrape data from a particular webpage. It has a total of 361 pages and 10 data rows on each page (total of 3610 data points). However, what I get is only 3260 data points.
Normally the process works fine and the event log looks like this:
[Click to Paginate] Waiting for Ajax to load...
[Click to Paginate] Ajax loaded.
Executing [Loop Item] variable list
[Loop Item] executing loop item #1
[Extract Data] Data successfully extracted
[Loop Item] executing loop item #2
[Extract Data] Data successfully extracted
[Loop Item] executing loop item #3
Executing [Loop Item] variable list
...
[Loop Item] executing loop item #10
[Extract Data] Data successfully extracted
[Loop Item] executed 10 times, exiting loop now.
Paginate
However, I notice that on multiple occasions, the event log shows something like this.
[Click to Paginate] Waiting for Ajax to load...
[Click to Paginate] Ajax loaded.
Executing [Loop Item] variable list
[Loop Item] executing loop item #1
[Loop Item] executed 1 times, exiting loop now.
I have tried to adjust different wait time but did not work. Anyone knows how to fix this would be much appreciated. Thanks!!!
