How to retrieve all Twitter users with TweetSharp using cursors?

Question

I'm using TweetSharp in a WPF .NET 4.0 C# app and using an authenticated Twitter service object. I'm having issues iterating through a list of followers to retrieve every single profile. I'm using the following code:

TwitterCursorList<TwitterUser> followers = twitterService.ListFollowersOf(userID, -1);
while (followers != null)
{
    foreach (TwitterUser follower in followers)
    {
        //Do something with the user profile here
    }
    followers = twitterService.ListFollowersOf(userID, (long)followers.NextCursor);
}

I see a weird behavior when I use this against my own account, which has 1271 followers as of this writing. The 1st time I run the code, I get a list of 100 followers, and on the next ListFollowersOf call, followers is null and the loop ends.

Here is the weird part: If I run this code again, whether in the same app instance, or if I stop Visual Studio and restart, doesn't matter, I get one extra ieration and now I get 200 followers back. If I do this trick again, now I get 300 followers back before I get a null, and so on. I repeated this many times until one call of the code above returned all 1271 followers.

The really weird thing is this eventually resets. I think it's tied to the Twitter API limit reset time, but I have not verified that. I'll see if this coincides with the API reset. Once the reset happens, I only get 100 followers, then 200, and so on.

I have reviewed TweetSharp unit tests and the following posts and they did not work for me:

Did you try `while (followers.NextCursor != null)`? Your while loop stops when `followers` is `null`, yet the other examples in the questions you linked stop when the `NextCursor` is `null`. — Kiril, May 24 '12 at 22:02
I have. followers.NextCursor returns a long?, and the issue was that once followers became null (and it does, as explained above), I would get an null reference exception thrown. — ActiveNick, May 25 '12 at 12:59
One of the [tests provided with the source](https://github.com/danielcrenna/tweetsharp/blob/master/src/net40/TweetSharp.Next.Tests/Service/TwitterServiceTests.cs) checks if `NextCursor` is `null` and you're casting it to long, so it does seem like it's a reference of some sort (I don't have the code to check it). Here is the check: `Assert.IsNotNull(followers.NextCursor);` You can still check if `followrs` is `null` and don't reference it if it's `null`, but it looks like your while loop should keep going until `NextCursor` is `null`... at least that's how it looks from the other examples. — Kiril, May 25 '12 at 14:31
I initially tried not casting to long but VS was throwing a compiler error "cannot convert from 'long?' to 'long'". I also get another compiler error "The best overloaded method match for 'TweetSharp.TwitterService.ListFollowersOf(int, long)' has some invalid arguments". — ActiveNick, May 25 '12 at 14:48
If `NextCursor` can be checked for `null` it's either a reference type or it's a nullable value type. Value types, like `long`, cannot be `null` unless the field/property/variable is declared as a nullable (e.g. `long?`). If the method expects a `long` you have to cast `NextCursor` to `long` regardless if it's a reference type or a nullable `long?`. — Kiril, May 25 '12 at 15:07
I checked every single overload and all cursor parameters expect a long, NOT a nullable long?, so I guess my cast is required. I don't understand how prior examples in these other threads ever worked. It could be an older version of TweetSharp that allowed that. Regardless, the mystery remains: Why would the method return a null after 100 followers, and then 100 more if I run the SAME code again. — ActiveNick, May 25 '12 at 15:22
Check my answer... I think that your code should look like that based on the examples I've seen. I don't have the library, so I can't run the code, but try it and let me know if it works. — Kiril, May 25 '12 at 15:32

score 1 · Answer 1 · answered Jan 08 '13 at 07:02

I use something like this:

public static List<TwitterUser> GetFollowers(this TwitterService svc)
{
    List<TwitterUser> ret = new List<TwitterUser>();

    var followers = svc.ListFollowers(-1);
    ret.AddRange(followers);
    while (followers.NextCursor != null && followers.NextCursor.Value > 0)
    {
        followers = svc.ListFollowers(followers.NextCursor.Value);
        ret.AddRange(followers);
    }
    return ret;
}

and then

var f = svc.GetFollowers();

score 0 · Answer 2 · answered May 25 '12 at 15:08

I don't have the library, but based on the examples it looks like you should have something like this:

TwitterCursorList<TwitterUser> followers = twitterService.ListFollowersOf(userID, -1);
while (followers.NextCursor != null)
{
    if(followers != null)
    {
        foreach (TwitterUser follower in followers)
        {
            //Do something with the user profile here
        }
    }
    followers = twitterService.ListFollowersOf(userID, (long)followers.NextCursor);
}

Again, I can't run the code since I don't have the library (too lazy to download it too), but give it a shot and let me know if it works.

score 0 · Answer 3 · answered Mar 20 '13 at 23:44

If you're getting anything back it's because your code works just fine.

It's easy to find out what's going wrong and then begin to fix your problems accordingly. I suspect that you're correct and that you are being rate-limited by Twitter.

Add this to your code after your while-loop:

if(followers == null) {
    Console.WriteLine("Error " + twitterService.Response.Error);
}

In fact, any time something returns null you can check the Response object's Error to see what it is TweetSharp thinks went wrong. See this.

How to retrieve all Twitter users with TweetSharp using cursors?

3 Answers3