How to avoid rate-limiting 429 error in Twython

Question

I've created a function designed to run through a column of Twitter handles pandas dataframe, yet it always seems to hit rate-limiting error after just 14 calls.

Here's the code.

def poll_twitter(dfr):
    followers  = twitter.get_followers_ids(screen_name = dfr['handle'])
    time.sleep(5)
    print "looping..."
    return len(followers['ids'])

df[datetime.datetime.today()] = df.apply(poll_twitter, axis=1)

Here's the error

TwythonRateLimitError: (u'Twitter API returned a 429 (Too Many Requests), Rate limit exceeded'

The list is only 100 handles so I assumed there would be plenty of calls available.

What's the way of fixing it?

score 3 · Accepted Answer · answered Dec 10 '13 at 10:30

3

Twitter GET followers/ids endpoint in API 1.1 version has 15 requests/per window (15 mins) limit, i.e. about 60 requests per hour.

Note also, that it also returns up to 5000 ids per request, so you have to issue more requests for highly followed users. For example only Barack Obama followers list will take 40434976/(5000*60*24) = 5.62 days to be loaded.

answered Dec 10 '13 at 10:30

alko

46,136
12
94
102

Thank you. I've ended up using requests to simply grab the summary stats I needed without authentication. – elksie5000 Dec 10 '13 at 13:13
1

@elksie5000 if you only need counts, look at [users/lookup endpoint](https://dev.twitter.com/docs/api/1.1/get/users/lookup), it returns stats for up to 100 users, followers_counts included. – alko Dec 10 '13 at 13:15
Even better :) Thank you again – elksie5000 Dec 10 '13 at 13:18

score 0 · Answer 2 · answered Jul 06 '20 at 20:07

0

You can use Twython's get_lastfunction_header('x-rate-limit-remaining').

answered Jul 06 '20 at 20:07

yihlamur

382
1
5
17

How to avoid rate-limiting 429 error in Twython

2 Answers2