0

I am trying to download the list of friends of a certain user on Twitter using rtweet.

If I try to download the friends list of a user who has less than 5000 friends, I get the expected result - a list of the user's friends.

However, if I try to download the friends list of a user who has more than 5000 friends I hit the API limit, so I have to use the page parameter with the get_friends() function as detailed here: https://rtweet.info/reference/next_cursor.html

I am using @jimmyfallon as an example because he has more than 5000 Twitter friends.

f1 = get_friends("@jimmyfallon", retryonratelimit = TRUE)$user_id
Sys.sleep(60*15) # Sleep for 15mins
f2 = get_friends("@jimmyfallon", retryonratelimit = TRUE, page = next_cursor(f1))$user_id

The expected output of the code above would be that f1 contains the first 5000 friends and f2 contains the remaining lot of friends.

My actual result is f1 contains the first 5000 friends and f2 contains a 0x0 tibble (no data).

The strange thing is, sometimes I actually do get the expected output for f2 but most times it does not give the expected output.

I have tried increasing the sleep time to 30 minutes, used a different API key, removing the @ in get_friends() and executing on a different network.

I am using R 3.6.1 and rtweet_0.6.9, running in RStudio.

jkrix
  • 15
  • 8
  • Try `f2 = get_friends("@jimmyfallon", n = 10000, page = -2, retryonratelimit = TRUE)$user_id` to get more of the IDs. between that and your first one I got about 9,000. – Jon Spring Oct 14 '19 at 02:28
  • @JonSpring I just tried your solution, for me, `f2 = get_friends("@jimmyfallon", n = 10000, page = -2, retryonratelimit = TRUE)$user_id` looks to be pulling in the first 5000 friends rather than the next lot. – jkrix Oct 14 '19 at 02:40
  • Your first code, when I run it now, produces 5000 starting with 44393707, while my code produces 5000 starting with 2162125610. Binding them together and counting yields 8,274 id's in just one of the two, and 863 overlapped in both, for 9,137 total, which is exactly as many followers as he has right now. – Jon Spring Oct 14 '19 at 03:13
  • @JonSpring Thank you, your solution seems to be working fine. To get the full 9137 friends stored in the one object I have done my first line and then your solution next, after I have done `strip = (5000-which(f1 == f2[1]))+1` and combined them together with `f = c(f,f1[-(1:strip)])` – jkrix Oct 14 '19 at 09:20

0 Answers0