1

I'm trying to write a program that will retrieve all of the tweets the user has seen on their twitter home timeline (ie, from people their following, as they would see at twitter.com). I realise this is a lot of data, and the rest API has limitations.

What would be the be best way to do this? Slowly retrieve the last 200 or whatever the limit is tweets, keeping in mind the 350 requests per hour limit? Or is there some hard limit to how far back I can go even with that?

The streaming API only streams from current point on I believe, so I don't think this is an option. This is a personal project so I can't pay very much for any elevated access or anything.

RodH257
  • 3,552
  • 5
  • 36
  • 46

1 Answers1

3

Yes, there is a limit to how far back you can go:

Clients may access a theoretical maximum of 3,200 statuses via the page and count parameters for the user_timeline REST API methods. Other timeline methods have a theoretical maximum of 800 statuses. Requests for more than the limit will result in a reply with a status code of 200 and an empty result in the format requested. Twitter still maintains a database of all the tweets sent by a user. However, to ensure performance of the site, this artificial limit is temporarily in place.

Source: http://dev.twitter.com/pages/every_developer

As you mentioned, you will need to go page by page through each of the 200 max results that come back until you hit that limit and get the empty result set, making sure not to hit the 350 requests per hour. There might also be gaps depending on how many tweets the user has on their timeline.

onteria_
  • 68,181
  • 7
  • 71
  • 64