Maximizing Twitter's rate limit to log followers

Question

Twitter API has a rate limit of 15 requests per 15 minutes.

I am trying to use python to run through and compile a list of my followers, but for some reason, the rate limit maxes when only 300 followers are logged. I've read that the max should be closer to 5,000. Where am I going wrong?

If I have thousands, or even more than 1 million followers and am trying to print a list of each follower, how can I maximize the number of followers that Python/twitter returns before hitting the rate limit?

Here is the code I have so far:

followers = tweepy.Cursor(client.followers, id=screenName)
for follower in followers.items():
    info=[]
    name =follower.name
    screen_name = follower.screen_name

    userId = userId + 1

    info.append(userId)
    info.append(name)
    info.append(screen_name)

    csvFile = open('followers.csv','a')
    newFile =csv.writer(csvFile) #imported csv
    newFile.writerow(info)
    #close file
    csvFile.close()

score 4 · Answer 1 · edited Jun 20 '20 at 09:12

When you access to client.followers in tweepy, you are actually performing this twitter request GET followers list. In this request, you have two limitations:

(1) number of request per 15-min window:

Requests / 15-min window (user auth) 15

Requests / 15-min window (app auth) 30

(2) number of followers that you can get on each request, see parameter count:

The number of users to return per page, up to a maximum of 200. Defaults to 20.

Thus, just to put an example, imagine for example that all users have 200 followers, you could get the followers of 15 users in a 15 min window, and you would get 3000 followers in total. Another example, if a user has 3000 followers, you would need 15 request to fetch them all, and then you would use the full 15 min window. (I assume you use user auth).

On the other hand, if your users had only 1 follower, you would get 15 followers on each 15 min window.

The number 5000 you mention is for another twitter request GET followers ids, which in tweepyis client.followers_ids. In this one, you don't get the full information of the user, only the id, but the rate limits are less strict:

(1) number of request per 15-min window:

Requests / 15-min window (user auth) 15

Requests / 15-min window (app auth) 15

(2) number of follower ids that you can get on each request, see parameter count:

Specifies the number of IDs attempt retrieval of, up to a maximum of 5,000 per distinct request.

So, with this request you could get up to 15*5000 = 75000 followers per window (however, I insist, only the id's).

Hope it helps.

Maximizing Twitter's rate limit to log followers

1 Answers1