How to create an efficient for loop to resolve the rate limit issue with twitteR?

Question

I am quite new to TwitteR and the concept of for loop. I have come across to this code to get the followers and profiles.

This code below works fine. Not entirely sure if retry on rate limit should be set for such a long time.

#This extracts all or most followers. 
followers<-getUser("twitter_handle_here")$getFollowerIDs(retryOnRateLimit=9999999)

This code below is the for loop to get the profiles.

However, I think there should be a way to use length(followers) and getCurRateLimitInfo() to better contruct the loop.

My question is that if the length(followers) = 40000 and the ratelimit = 180, then how to construct the loop to sleep with the right amount of time and to get all 40000 twitter profiles?

Any help would be much appreciated.

#This is the for loop to sleep for 5 seconds.
#Problem with this is it simply sleeps for X seconds
for (follower in followers){
  Sys.sleep(5)
  followers_info<-lookupUsers(followers)
  followers_full<-twListToDF(followers_info)
  }

Answer 1

Here is some code I had written for a similar purpose, First you need to define this function stall_rate_limit:

stall_rate_limit <- function(limit) {

  # Store the record of all the rate limits into rate
  rate = getCurRateLimitInfo()
  message("Checking Rate Limit")

  if(any(as.numeric(rate[,3]) == 0)) {

    # Get the locations of API Calls that are used up
    index = which(as.numeric(rate[,3]) == 0)

    # get the time till when rates limits Reset
    wait = as.POSIXct(min(rate[index,4]),     ## Reset times in the 4th col
                      origin = "1970-01-01",  ## Origin of Unix Time
                      tz = "US/Mountain")     ## Replace with your Timezone

    message(paste("Waiting until", wait,"for Godot to reset rate limit"))
    # Tell the computer to sleep until the rates reset
    Sys.sleep(difftime(wait, Sys.time(), units = "secs"))

    # Set J = to 0
    J = 0
    # Return J as a counter
    return(J)

    } else {

    # Count was off, Try again
    J = limit - 1
    return(J)

    }
}

Then you can run your code something like this:

callsMade = 0    ## This is your counter to count how many calls were made
limit = 180      ## the Limit of how many calls you can make
for(i in 1:length(followers)){

  # Check to see if you have exceeded your limit
  if(callsMade >= limit){

    # If you have exceeded your limit, wait and set calls made to 0 
    callsMade = stall_rate_limit(limit)

  }

  ### Execute your Code Here ... ###

  callsMade = callsMade + 1  # or however many calls you have made
}