0

I am trying to write an R program using the package twitteR, but I keep running into the rate limit. I hacked this together to try to sleep until the rate limit was finished. The count<10 was intended to keep the program moving if there were no returnable values with that string. searchedTweets is a global variable that holds all the results. How can I make this more "R-like"? Specifically how can I correctly use the retryOnRateLimit parameter for this? How can I better handle not finding any tweets? 

searchString<-c('#BringBackOurGirls','#YesAllWomen','#HeforShe','#NotAllMen','#IceBucketChallenge','#OccupyCentral','#Ferguson','#Gamergate','#YouOKSis','#BlackLivesMatter','#StopGamerGate','#TweetLikeANeurotypical','#CancelColbert','#UmbrellaRevolution','#AmINext','#IfTheyGunnedMeDown','#WhyIStayed','#DudesGreetingDudes','#GirlsLikeUs','#MensRights','#MGTOW','#RedPill','#CheerstoSochi','#NotOneMore','#Not1More','#DonLemonReporting','#CrimingWhileWhite','#LoveIsLove','#PrayForParis','#LoveWins','#IStandWithAhmed','#IStandWithPlannedParenthood','#ShoutYourAbortion','#ProLife','#Oromoprotests','#Kony2012','#SosBlakaustralia')
searchedTweets<<-NULL
lapply(searchString,function(y){
r_tweets<-NULL
count<-0
while(length(r_tweets)<=0 && count<10){
r_tweets<-searchTwitter(y, n=50, lang="en-US", since=NULL, until=NULL,
                        locale=NULL, geocode=NULL, sinceID=NULL, maxID=NULL,
                        resultType=NULL, retryOnRateLimit=0)
Sys.sleep(60*2)#seconds
count<-count+1
cat("count: ",count," on term: ",y,"\n")
}
if(count<10){
searchedTweets<<-rbind.data.frame(searchedTweets,r_tweets)
sources<-sapply(r_tweets,function(x)x$getStatusSource())
sources<-gsub("</a>","",sources)
sources<-strsplit(sources,">")
sources<-sapply(sources,function(x)ifelse(length(x)>1,x[2],x[1]))
source_table=table(sources)
pie(source_table[source_table>10])
cat("###term: ",y," done!\n")
#pie(source_table)
}else{cat("couldnt use term: ",y)}
})
Rilcon42
  • 9,584
  • 18
  • 83
  • 167
  • Possible duplicate of [Rate Limit in Twitter REST API - a clarification needed](http://stackoverflow.com/questions/23267464/rate-limit-in-twitter-rest-api-a-clarification-needed) – Yuri Schimke Jan 25 '16 at 01:50
  • Your link does provide a useful source, but the questions are not duplicates. The linked question deals with the theoretical implementation of the limit. My question deals with how to programatically check if the limit is exceeded. – Rilcon42 Jan 25 '16 at 21:14

1 Answers1

0

You should be able to be more methodical about timing your requests to not get rate limited or how long to sleep for your rate limit to reset. The response headers will tell you the rate limit.

$ oksocial -i 'https://api.twitter.com/1.1/search/tweets.json?q=%40twitterapi'
HTTP/1.1 200 OK
...
x-rate-limit-limit: 180
x-rate-limit-remaining: 179
x-rate-limit-reset: 1453674071

This will give your limit and remaining requests before the next reset.

$ date -r 1453674071
Sun 24 Jan 2016 14:21:11 PST
Yuri Schimke
  • 12,435
  • 3
  • 35
  • 69