linq to twitter limited result to a week, how to exceed it?

Question

The following code is to get all tweets in a hashtag. if you tried different tags you'll have different result because of the amount of tweets in the last week. the problem is not related to the limit rate time of 15min. I have tried to delay the function. How can I exceed this limit of the first week to all the tweets until I get the first tweet in this hashtag?

 private List<Status> statusList= new List<Status>();
        private ulong sinceID=1;
        private ulong maxID;        

        public void GetMostRecent200HomeTimeLine()
        {
            var twitterContext = new TwitterContext(authorizer);
            var tweets = from tweet in twitterContext.Status
                         where tweet.Type == StatusType.Home && tweet.Count == 200
                         select tweet;
            currentTweets = tweets.ToList();
        }


        public bool TAsyncSearchTag(string tag)
        {
            var twitterContext = new TwitterContext(authorizer);

            Search searchResponse;
            try
            {
                do
                    {    
                    searchResponse =
                        (from search in twitterContext.Search
                         where search.Type == SearchType.Search &&
                         search.ResultType == ResultType.Mixed &&
                         search.Query == tag &&
                         search.Count == 100 &&
                         search.SinceID == sinceID &&
                         search.MaxID == maxID
                         select search)
                         .FirstOrDefault();

                    if (searchResponse != null && searchResponse.Statuses.Count > 0)
                            {
                                var tweets = from tweet in searchResponse.Statuses
                                         select tweet;
                                statusList.AddRange(tweets);
                                maxID = searchResponse.Statuses.Min(x => x.StatusID) - 1;
                            }

                        } while (searchResponse.Statuses.Count != 0);
                return true;      
                }
            catch (Exception e)
                {
                MessageBox.Show(e.Message);
                return false;
                }
        }

        public List<Status> GetAllTweets(string tag)
        {
            var twitterContext = new TwitterContext(authorizer);
            var searchResponse =
                (from search in twitterContext.Search
                 where search.Type == SearchType.Search &&
                 search.ResultType == ResultType.Mixed &&
                 search.Query == tag &&
                 search.SinceID == sinceID &&
                 search.Count == 100
                 select search)
                 .FirstOrDefault();

            var tweets = from tweet 
                             in searchResponse.Statuses
                             select tweet;
            statusList.AddRange(tweets);
            maxID = searchResponse.Statuses.Min(x => x.StatusID) - 1;

            bool tester = TAsyncSearchTag(tag);

            MessageBox.Show("done   " + statusList.Count.ToString() + "  -  " + statusList.Last().StatusID.ToString());

            if (statusList != null)
                return statusList;

            return null;
        }

Update, I have found that twitter limit their search to 1 week maximum, to fetch more than that. Other methods must be used by using users or related tweets. — Feras, Jul 02 '15 at 14:31

score 0 · Answer 1 · answered Jun 30 '15 at 17:11

I am having some issue trying to determine what the real problem is, but I am assuming you want to get the first result and are having issue. An understanding of LINQ is

LINQ uses deferred execution to iterate collections. It grabs the result set when you start iterating. It then applies filters. LINQ to SQL is a slight exception, as it has a query parser built in. In LINQ to SQL, it actually write the query, so the set will be somewhat filtered in the call. But, in LINQ to {choose your favorite service here} you will get all of the results before you can get the first result.

In other words, your problem IS likely related to the limit. Despite your filtering to a small number of results or a single result in LINQ, all of the results are being requested and then LINQ is filtering the entire result set.

I would separate the grabbing of data from the LINQ filter and you should find things work much better

I'm trying to get all the tweets in a certain hashtag. I agree with you Linq is filtering my request to the last week tweets only. I want to know how can I got more than this week actually all the tweets in this hashtag. — Feras, Jul 01 '15 at 18:24
The problem is you are producing a list of potentially millions of tweets, although the user of your application cannot possibly consume them. You are better to filter the result set first and then organize it, which may be done with, or without, LINQ. — Gregory A Beamer, Aug 06 '15 at 18:38

linq to twitter limited result to a week, how to exceed it?

1 Answers1