0

Below is my working code to get twitter followers for certain accounts (@hudsonci in this case).

My issue is the time that it is taking to pull in all of these followers. This account specifically has approx 1,000 followers ... I can only get to 300 at a time with the rate limiting restrictions. So, it is taking > an hour to get all the followers for this account. I can imagine this will become a huge pain in the ass for large accounts.

I am looking for some suggestions for how I can improve this. I feel like I am not taking full advantage of the pagination cursor, but I can't be sure.

any help is appreciated.

#!/usr/bin/env python
# encoding: utf-8

import tweepy 
import time 

#Twitter API credentials
consumer_key = "mine"
consumer_secret = "mine"
access_key = "mine"
access_secret = "mine"


#authorize twitter, initialize tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)

def handle_errors(cursor):
    while True:
        try:
            yield cursor.next()
        except tweepy.TweepError:
            time.sleep(20 * 60)

for user in handle_errors(tweepy.Cursor(api.followers,screen_name='hudsonci').items()):
     print user.screen_name
hansolo
  • 903
  • 4
  • 12
  • 28

2 Answers2

1

As per the Twitter documentation for followers you need to use the count parameter.

Specifies the number of IDs attempt retrieval of, up to a maximum of 5,000 per distinct request.

So, adding count=5000 should help you.

Terence Eden
  • 14,034
  • 3
  • 48
  • 89
  • Changed the script to include `for user in handle_errors(tweepy.Cursor(api.followers,screen_name='hudsonci', count= 5000).items()):` And I am still only getting 300 followers at a time. Is the count in the wrong place? I feel like I am missing something very simple here. – hansolo Sep 29 '16 at 13:20
  • The limit for the followers API is 200, so you should be able to get 200 * 15 = 3000 in any 15-minute interval – Cory Nov 12 '20 at 06:44
0

You are getting 300 followers at a time because getting the followers object (as opposed to IDs only) has a page limit 20. With 15 requests per window, that comes out to be 300 followers.

Here are the docs for followers: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-list

user3243179
  • 1
  • 1