Twitter error code 429 with Tweepy

Question

I am trying to create a project that accesses a twitter account using the tweepy api but I am faced with status code 429. Now, I've looked around and I see that it means that I have too many requests. However, I am only ever for 10 tweets at a time and within those, only one should exist during my testing.

for tweet in tweepy.Cursor(api.search, q = '@realtwitchess ',lang = ' ').items(10):
                        try:
                                text = str(tweet.text)
                                textparts = str.split(text) #convert tweet into string array to disect
                                print(text)

                                for x, string in enumerate(textparts): 
                                        if (x < len(textparts)-1): #prevents error that arises with an incomplete call of the twitter bot to start a game
                                                if string == "gamestart" and textparts[x+1][:1] == "@": #find games
                                                        otheruser = api.get_user(screen_name = textparts[2][1:]) #drop the @ sign (although it might not matter)
                                                        self.games.append((tweet.user.id,otheruser.id))
                                        elif (len(textparts[x]) == 4): #find moves
                                                newMove = Move(tweet.user.id,string)
                                                print newMove.getMove()
                                                self.moves.append(newMove)
                                if tweet.user.id == thisBot.id: #ignore self tweets
                                        continue

                        except tweepy.TweepError as e:  
                                print(e.reason)
                                sleep(900)
                                continue
                        except StopIteration: #stop iteration when last tweet is reached
                                break

When the error does appear, it is in the first for loop line. The kinda weird part is that it doesn't complain every time, or even in consistent intervals. Sometimes it will work and other times, seemingly randomly, not work.

We have tried adding longer sleep times in the loop and reducing the item count.

score 44 · Answer 1 · answered Apr 02 '18 at 01:22

44

Add wait_on_rate_limit=True on the API call like this:

api = tweepy.API(auth, wait_on_rate_limit=True)

This will make the rest of the code obey the rate limit

answered Apr 02 '18 at 01:22

J. Gandra

553
6
12

Giordano · Accepted Answer · 2023-01-11T16:16:28.640

25

You found the correct information about error code. In fact, the 429 code is returned when a request cannot be served due to the application’s rate limit having been exhausted for the resource.(from documentation)
I suppose that your problem regards not the quantity of data but the frequency.

Check the Twitter API rate limits (that are the same for tweepy).

Rate limits are divided into 15 minute intervals. All endpoints require authentication, so there is no concept of unauthenticated calls and rate limits. There are two initial buckets available for GET requests: 15 calls every 15 minutes, and 180 calls every 15 minutes.

I think that you can try to use API in this range to avoid the problem

Update
For the latest versions of Tweepy (from 3.2.0), the wait_on_rate_limit has been introduced.
If set to True, it allows to automatically avoid this problem.

From documentation:

wait_on_rate_limit – Whether or not to automatically wait for rate limits to replenish

edited Jan 11 '23 at 16:16

answered Jan 23 '17 at 21:26

Giordano

5,422
3
33
49

Probably yes, but you have to set up sleeps in order to respect the rate limits – Giordano Jul 18 '19 at 07:11
I put sleep(10) and still I get the same issue :/ – mina Jul 18 '19 at 07:14
Ya we need to save the tweets and hit the api only if needed :) – Praveen Kumar Palai Apr 17 '20 at 22:27
Why am i getting rate limited then? I have used 3 of 2,000,000 streamed tweets. – Jridyard Dec 23 '22 at 17:21
@Jridyard sorry but it is not very clear what you are asking, can you please post a new question with all the details? thanks – Giordano Dec 27 '22 at 06:59
1

@Giordano I was mostly venting. I figured it out eventually. It is because, although it's not clear, Twitter limits you to 25 "rules" and each influencer you try to stream tweets from counts as 1 "rule". So I was not allowed to stream from the 300 as I had intended. Oh well, 3 weeks of work well spent I guess. – Jridyard Dec 27 '22 at 19:47

score 3 · Answer 3 · answered Apr 28 '21 at 09:09

3

api =tweepy.API(auth,wait_on_rate_limit=True,wait_on_rate_limit_notify=True)

this should help for setting rate

answered Apr 28 '21 at 09:09

roropanda

227
3
13

Twitter error code 429 with Tweepy

3 Answers3