Twitter Search API - since_id and max_id to generate more results

Question

I'm using Abraham twitter library to search for tweets. All is going fine, but the problem is, my search returns more than 100 results.

Is there an easy way to loop through, like paging using the newer v1.1 of the API?

I need to be able to get all of the tweets via the search from a particular search, then loop through each of the results to create an array

This is what I currently have:

$connection = new Abraham\TwitterOAuth\TwitterOAuth(
   ‘KEY',
   ‘KEY',
   ‘KEY',
   ‘KEY'
);

$statuses = $connection->get("search/tweets”,
    array(
        "q"                 => ’something',
        "result_type"       => "recent",
        "include_entities"  => true,
        "count"             => 100
    )
);
// possibly put into a loop - paging

var_dump($statuses->statuses) gives me an array of objects (100) So lets say my search term returned 298 results. Based on my code above, I’d have 3 pages so to speak.

I was wondering what the best approach for this is?

The search terms I’m using a some what popular, maybe 100 a minute. I'm looking at running this via a cronjob, every hour, lets say, so I could have 600-1000 tweets in my array

I’ve seen the since_id and max_id, but not sure how I’d use these in my application.

I’m looking to store the tweets in a database, so I could potentially use the since_id and/or ‘max_id’

Thanks

score 2 · Answer 1 · answered Jan 23 '16 at 14:52

2

search/tweets (without either since_id or max_id) returns the latest tweets ordered chronologically beginning with the most recent tweet. So, you will need to save the id of the last tweet returned. To get the next "page" of tweets, simply repeat the request, but also include max_id set to the value of id you saved.

Based on your code snippet, I think the request would look like this:

$statuses = $connection->get("search/tweets”,
    array(
        "q"                 => ’something',
        "result_type"       => "recent",
        "include_entities"  => true,
        "count"             => 100,
        "max_id"            => <VALUE OF LAST ID>
    )
);

answered Jan 23 '16 at 14:52

Jonas

3,969
2
25
30

Thanks for the feedback. I have a `since_id` that I'm using to get the initial tweets (100). Is there an easy way to then use the `max_id` form this query to get the subsequent pages? i.e tweets 101-199 and then 200-298 into an array? – sipher_z Jan 25 '16 at 14:24
After your first query do the following: 1) remove `since_id` from your query, 2) add "`max_id` = ." – Jonas Jan 25 '16 at 16:02
Great! I think i have a working version now. I am using the `since_id` in the first request, then using the subsequent `max_id` to get the next set of tweets. This is all in a while loop and seems to work! – sipher_z Jan 26 '16 at 11:30
Hi Jonas, I think I'm almost there. Would you be able to check over my code? – sipher_z Jan 27 '16 at 13:53
@sipher_z Sorry, I am not a PHP programmer. – Jonas Jan 27 '16 at 17:09
Do you have an example in Python? – sipher_z Jan 28 '16 at 02:16
@sipher_z, Do u working version to navigate through the tweet page in php? – SESN Feb 22 '17 at 16:53

Twitter Search API - since_id and max_id to generate more results

1 Answers1