1

I am trying to find the top most followers based on the number of tweets about a brand named Maybelline on Twitter. The brand has about 600K followers and when I try to retrieve them the code keeps running for hours. So is there an efficient way to do this? I am using the below code after setting up twitter authentication. I want all the followers ( top 50) who tweeted the most about Maybelline.

user<-getUser('Maybelline')
user$toDataFrame()
followers<-user$getFollowers()

Thanks

Abhinav
  • 501
  • 1
  • 5
  • 17
  • I've edited my answer to give you an alternative that would take most at most 8 hours assuming the twitter R package supports these methods/calls. – metame Feb 14 '17 at 21:24

1 Answers1

5

While working with the Twitter API, it's useful to familiarize yourself with their limits. You have two main limits for a GET request, one is a Rate Limit (how many requests you can make in a 15 minute timeframe) and the other is the limit of how many results a certain call gives you.

In your scenario, you are using the GET followers/list endpoint from their API. You can read the docs for that here. That endpoint returns a list of followers and is limited to 20 followers per request and 15 request per 15 minutes, meaning that in a 15 minute timeframe, you can only retrieve 15*20 = 300 users. So to retrieve 600K followers that would take a very long time ( 30K minutes = 500 hours = ~21 days ).

It would be more efficient to use the GET followers/id which returns up to 5K user ids with each request with the same 15 request per 15 minute rate limit. Twitter API reference here. You can use this in conjunction with the Twitter GET users/lookup which returns up to 100 users per request and has a rate limit of 900 requests per 15 minutes. This means it would take 2 hours (at 75K users per 15 minute increment) to get 600K followers id's. And less than 2 hours to get the user objects ( at 90K users per 15 minutes ).

The rate limits can change depending on how the package you are using does authentication. If you are logging in as a Twitter user with your credentials, then the above rate limits are correct. If you are using only application credentials, then getting the followers will take 3x longer as users/lookup has a rate limit of 300 requests in that case or 30K users per 15 minutes. This answer has some good information on rate limits.

Community
  • 1
  • 1
metame
  • 2,480
  • 1
  • 17
  • 22