Twitter Rate Limiting IP/OAuth Concerns

Question

I have a series of webapps that collects all terms relating to a subject using the Public Streaming API. So far, I've been taking a very, very arduous route of creating a new account for each stream, setting up a new Twitter application on that account, copying the OAuth tokens, spinning up a new EC2 instance, and setting up the stream.

This allows me to have the streams coming from multiple different IPs, OAuth generation is easy with the generator tool when you create an app, and because they are each in different accounts I don't meet any account limits.

I'm wondering if there's anything I can do to speed up the process, specifically in terms of EC2 instances. Can I have a bunch of streams running off the same instance using different accounts?

Probably yes. On the bottom of the linked page there is worning about abusing Publis Streaming API which refer to one account and one IP. So as long as you use different accouts you should be on the green side even though you connect from one IP. However you should really contact Twitter for more info, and exact quotas. — przemo_li, Sep 16 '12 at 12:18

score 4 · Accepted Answer · answered Sep 16 '12 at 12:15

4

If you run multiple consumers from a single machine you may be temporarily banned, repeated bans may end up getting you banned for longer periods. At least, this happened to me a few times in the past.

What I had found at the time was:

same credentials, same ip -> block/ban
different credentials, same ip -> mostly ok, but banned from time to time
different credentials, different ip -> ok

This was a few years ago, so I am not sure the same is still true, but I'd expect twitter to have tightened the rules, rather than having relaxed them.

(Also, I think you're infringing their ToS)

answered Sep 16 '12 at 12:15

riffraff

2,429
1
23
32

Interesting results... I have found that different credentials, same IP always works with 15 messages per 15 minutes. I agree about their terms though... – Anthony Mason Jun 13 '16 at 15:34
1

my answer is a few years old, things might have changed :) – riffraff Jun 14 '16 at 17:59

Cesar Canassa · Answer 2 · 2012-09-20T15:18:33.637

You should check the new Twitter API version 1.1. It was released a few days ago and many changes were made on how the rates are calculated.

One big change is that the IP is completely ignored now. So you don't have to create many instances anymore (profit!)

From the Twitter dev @episod:

Unlike in API v1, all rate limiting is per user per app -- IP address has no involvement in rate limiting consideration. Rate limits are completely partitioned between applications.

Twitter Rate Limiting IP/OAuth Concerns

2 Answers2