0

I'm making a basic Sinatra application to display a user's Twitter mentions timeline using the Twitter gem. After logging in, I fetch his/her mentions. Something like:

get '/' do
  @mentions = Twitter.mentions_timeline

  erb :home
end

The issue with this is that I am making a call to the Twitter API every single time the user hits the homepage. I get rate limited and it's also not efficient, since I only want to re-fetch the mentions timeline, say, every 3 minutes. Or if it's simpler, I could just use the cache once I hit the rate limit.

After reading up, it seems like the best way to do this would be store this data in cache so I don't keep making API requests. How would I go about doing that?

tonic
  • 453
  • 1
  • 6
  • 21

2 Answers2

3

I do this by storing the tweets in redis. I am loading the tweets in a helper function, so that I can access the same cached copy in multiple routes.

All my code is on https://github.com/timmillwood/millwoodonline/blob/master/helpers/main.rb#L38 feel free to take what you need.

  • And, some memoization could reduce the number of requests even more. – Kashyap Dec 11 '12 at 11:08
  • thanks! few notes for newbie's like me stumbling upon this page: 1) you need to install redis, compile it and run the redis server, 2) you need to install the redis gem, 3) i had to convert my twitter timeline to json to store it – tonic Dec 11 '12 at 23:27
  • could you explain how were you able to store an Array in cache? converting it to json is OK but your gist shows you did not. – tonic Dec 12 '12 at 02:22
  • I am using RedisToGo so I don't host it myself, but connect to it using the redis gem. I think fetch the twitter feed and wrap the tweets in all the HTML needed, then cache the HTML in Redis rather than the raw Twitter data. Redis will only store plain text, so you will need to store a json string if you want raw data. I also store all my page HTML in Redis too http://www.millwoodonline.co.uk/blog/cut-average-response-time-by-three-quarters-using-redis –  Dec 12 '12 at 12:24
1

The obvious way would be to store the user's timeline in a database. Using something like MongoDB is pretty trivial, since the Twitter API returns json and you can just toss the tweets in there as they come. Before calling the Twitter API check the age of the most recent mention of the user, and don't call if it's relatively new (e.g. 30 minutes or less).

Diego Basch
  • 12,764
  • 2
  • 29
  • 24
  • You could also tuple the last access time with the cached data in your datastore or even put it in the JSON (though this means you'd have to parse the JSON to check whether you need to invalidate your cache). – Zach Dec 11 '12 at 03:28