0

EDITED: reasonable answer at comments.

I am looking for a way to check Twitter API rate limits before of making my consults but for the next snippet every time I call search , remaining is decremented twice instead of just once.

user = User.find_by_id(ID)
client = Twitter::REST::Client.new do |config|
  config.consumer_key = CONSUMER_KEY
  config.consumer_secret = CONSUMER_SECRECT
  config.access_token = user.access_token
  config.access_token_secret = user.access_token_secret
end

client.search("baeza")
puts Twitter::REST::Request.new(client, :get, 'https://api.twitter.com/1.1/application/rate_limit_status.json', resources: "search").perform
client.search("baeza")
puts Twitter::REST::Request.new(client, :get, 'https://api.twitter.com/1.1/application/rate_limit_status.json', resources: "search").perform

I am getting the next result:

{:rate_limit_context=>{:access_token=>"access_token"}, :resources=>{:search=>{:"/search/tweets"=>{:limit=>180, :remaining=>178, :reset=>1465385167}}}}
{:rate_limit_context=>{:access_token=>"access_token"}, :resources=>{:search=>{:"/search/tweets"=>{:limit=>180, :remaining=>176, :reset=>1465385167}}}}

I would appreciate hearing your thoughts. Thanks for reading!

Community
  • 1
  • 1
Daniel García Baena
  • 1,191
  • 4
  • 19
  • 33

1 Answers1

0

Can it be simply because call to rate_limit_status.json also counts? What if you do 

client.search("baeza")
client.search("baeza")
client.search("baeza")
puts Twitter::REST::Request.new(client, :get, 'https://api.twitter.com/1.1/application/rate_limit_status.json', resources: "search").perform

Will it return 176?

Update - call to rate_limit_status should not affect limits, but would be interesting to see the outcome of the suggested calls anyway.

Update 2

Gem fetches only first page by default, no matter of results count. Getting next page(s) is under your control. See here - fetch_next_page is private method, called by "each" method of enumerator here - so if you start iterating over search results and pass through all first page results. When you just made a request (called search) - it only fetches first page (one query).

Also, gem sets count parameter for API search request to 100 (max allowed) if not specified by default - here for constant and here for setting it

Pavel Bulanov
  • 933
  • 6
  • 13
  • I checked and no ... In addition, if you see the request it only asks for search resources. It seems to be something related with the size of the response – Daniel García Baena Jun 08 '16 at 12:25
  • So you mean if you search for something non-existing ('nostringeverputontwitter') it will reduce limit by 1? – Pavel Bulanov Jun 08 '16 at 12:30
  • It seems like if you search for a popular keyword where `search` returns a big array it is more *expensive* – Daniel García Baena Jun 08 '16 at 12:35
  • The thing is, that checking on https://dev.twitter.com/rest/tools/console directly doesn't prove it - searching for any term reduce limit by only 1 – Pavel Bulanov Jun 08 '16 at 12:36
  • What if you do a search query directly - request = Twitter::REST::Request.new(client, :get, '/1.1/search/tweets.json', q: "baeze"); results = request.perform; and then checking request.rate_limit.remaining? (this checks HTTP header x-rate-limit-remaining of the response) – Pavel Bulanov Jun 08 '16 at 12:39
  • I have been thoroughly reviewing the the API and search returns a maximum of tweets per page of 100. Therefore it should be necessary to make more than one request when result count is bigger than one hundred. For more info check `count` at: https://dev.twitter.com/rest/reference/get/search/tweets – Daniel García Baena Jun 08 '16 at 18:09
  • 1
    That is correct, but gem fetches only first page by default. Getting next page(s) is under your control. See [here](https://github.com/sferik/twitter/blob/77238b540edfbc76aa2f25bc4398c3dad8f55edb/lib/twitter/search_results.rb#L50) - fetch_next_page is private method, called by "each" method of enumerator [here](https://github.com/sferik/twitter/blob/77238b540edfbc76aa2f25bc4398c3dad8f55edb/lib/twitter/enumerable.rb#L13) - so if you start iterating over search results and pass through all first page results. When you just made a request (called search) - it only fetches first page (one query) – Pavel Bulanov Jun 08 '16 at 21:13
  • 1
    Also, gem sets count parameter for API search request to 100 (max allowed) if not specified by default - [here](https://github.com/sferik/twitter/blob/77238b540edfbc76aa2f25bc4398c3dad8f55edb/lib/twitter/rest/search.rb#L7) – Pavel Bulanov Jun 08 '16 at 21:43
  • Did you found out what is the reason? – Pavel Bulanov Jun 09 '16 at 15:13
  • Yes, it is because of what we discovered yesterday. You need to make several calls when search result is bigger than 100. I updated the question with the link to the comment. Thanks for your help! – Daniel García Baena Jun 09 '16 at 15:18
  • But it won't happen automatically, will it? I.e. search call itself will run just 1 request. And then iterating over results will produce further calls, right? – Pavel Bulanov Jun 09 '16 at 15:21
  • Exactly. Twitter gem iterates automatically and you will consume more than one request if it needs to iterate over more than 100 results. On the other hand, you will only consume one request if you directly call `Twitter::REST::Request.new(client, :get, '/1.1/search/tweets.json', q: "baeza")` but you will need to make more requests for more than 100 results too – Daniel García Baena Jun 09 '16 at 15:31