0

I have a list of approximately 1.500 twitter accounts (that may or may not have tweeted) for which I want to retrieve the last (max 100 tweets) every ~20 minutes. Considering the rate limits of Twitter API v.2, what is the most efficient way of doing this without hitting the rate limits (https://developer.twitter.com/en/docs/twitter-api/rate-limits)?

As far as I understand, there is no way of getting tweets from multiple users at the same time using https://api.twitter.com/2/users/<twitter id>/tweets and iterating through the 1.500 accounts to get the last tweets will make you hit the rate limit of ~900 requests per 15 minutes.

Is there a bulk request that can do this? Is adding them all to a Twitter list and get the latest tweets from there the only real option here?

I am needing this for a Node.js application but the issue is more about how to solve it at a Twitter API level.

Christoffer
  • 2,271
  • 3
  • 26
  • 57

1 Answers1

2

The Twitter search API is publicly available at /2/tweets/search/all. You can also use /2/tweets/search/recent.

Using this, you can search from tweets from multiple accounts at once using their OR operator:

(from:twitter OR from:elonmusk)

Returns:

{
  "data": [
    {
      "id": "1540059169771978754",
      "text": "we would know"
    },
    {
      "id": "1540058653155278849",
      "text": "ratios build character"
    },
    {
      "id": "1539759270501023744",
      "text": "RT @NASA: The landmark law #TitleIX opened up a universe of possibility for women, including Janet Petro, the 1st woman director of @NASAKe…"
    },
    // ...

Note, this has a more strict rate limit, and you will have a limit of how many characters you can use in your search (probably 512).

You can add extra fields like author_id from tweet.fields, if you need them.

If you cannot get by with this, then you may be able to combine API endpoints, since rate limits are applied per-endpoint. For example, search half via the searching endpoint, and the other half via the individual user endpoints.

If this still doesn't work, you're right (from everything that I've found), you will need to either:

  1. Increase your cache time from 20 minutes to something more 30-45 minutes
  2. Create a list
robere2
  • 1,689
  • 2
  • 16
  • 26
  • Thanks for helping out, although in the end it doesn't help much other than knowing that my other options are not great either. Unfortunately with lists, I figured out, you cannot get any older tweets than 600. But, what do. you mean by "increasing cache time"? Are you referring to the sleep between multiple requests? Or, is this a setting in Twitter API? – Christoffer Jun 27 '22 at 03:52
  • 1
    Your sleep time between requests. Instead of waiting 20 minutes, wait 30, or 45. Perhaps this doesn't make sense for your specific use-case, but without more information, it feels like it shouldn't be that big of a deal. – robere2 Jun 27 '22 at 16:28