I've created an app on twitter, and i can make a search and get results. For example when i searched #starbucks it has found 25.000 tweet but max paging limit is 100 record per page that means i have to navigate 250 page to insert theese tweets to my database. Twitter returns nextpage QueryString its fine so, results like this
- tweet
- tweet
- tweet ... 99.tweet 100.tweet
next page starts from 101 to 199..
i tried meta refresh to navigate automatically, after post form next page link comes to meta refresh url http://example.com?q=#starbucks&count=100&max_id=xxxxxxxxxxxx/">
and 60 secs after open next page, this process continue until last page.
my question is how can do this on linux with cronjob (vps,vds or hosting) i can make request for first page, because form working with get request, but can linux navigate pages automatically?
thank you.