8

I am using the following to cache a slow loading page using memcached:

caches_action :complex_report, :expires_in => 1.day

The controller action is protected by Devise authentication.

The page currently gets cached the first time a user requests it. Subsequent request that day are then pulled from the cache.

The problem with this is that the initial request takes 20-30 seconds to load. Is it possible to populate the cache in advance by way of a scheduled task?

Any suggestions much appreciated.

gjb
  • 6,237
  • 7
  • 43
  • 71

4 Answers4

6

Here is an expansion on the previous cron based solution which uses curl's ability to store cookies so that you can auth in one step and then use the cookie again as an authenticated user in the next step. So if you put these lines in a script called "prepare_cache.sh"

rm /tmp/cookiejar
curl --request POST -d "login=<username>" -d "password=<password>" -c /tmp/cookiejar http://yourwebpages.url/login
curl --request GET -b -c /tmp/cookiejar http://yourwebpages.url/page_to_cache
rm /tmp/cookiejar

replacing the login and password parameters with ones which match the variables used in your login form and obviously the urls to call. I'm removing the cookiejar before to make sure there isn't a file there already and removing it at the end to make sure there isn't a cookie floating about with access levels it shouldn't have.

Then you can call this script with the cron job:

*/15 * * * * /home/myname/prepare_cache.sh > /dev/null 2>&1

And hopefully that should work. Seemed to work for me when I tried it.

bjpirt
  • 236
  • 1
  • 1
  • That worked perfectly, thank you. A neat solution, which didin't require any modification to the existing code. – gjb Mar 30 '11 at 08:25
4

If it's the process of running the report and collecting results that is time-consuming, you could cache those results (in place of, or along-side action caching) using Rails.cache.write and Rails.cache.read.

Then, because you needn't worry about authentication or making requests to the server, the act of running the query and caching the results from a cron job would be considerably simpler.

idlefingers
  • 31,659
  • 5
  • 82
  • 68
  • I like this approach - cache the data that appears on the report, rather than the report itself, and then warm up that cache instead. If the report is frequently accessed, might even be worth doing both, but only warming up the cache of the back end data. – Paul Russell Mar 30 '11 at 07:16
  • Thank you for your suggestion. This would certainly help by caching the results of the complex query, however there is also some conditional formatting going on in the view (different table cell styles for different values). For this reason, I would prefer to cache the page rather than just the data. – gjb Mar 30 '11 at 08:17
4

Take a look at this gem:

https://github.com/tommyh/preheat

The gem is for preheating your Rails.cache.

From the documentation: This will "preheat" all your Rails.cache.fetch calls on your homepage. It is as simple as that!

    #app/models/product.rb
    def slow_method
      Rails.cache.fetch("product-slow-method-#{self.id}") do
        sleep 15
        Time.now
      end
    end

    #lib/tasks/preheat.rake
    namespace :preheat do
      desc "Preheat product caches"
      task (:products => :environment) do
        Preheat.it do
          Product.all.each do |product|
            app.get(app.products_path(product)) #or you could just call product.slow_method directly, whatever makes more sense
          end
        end
      end
    end

    #crontab -e
    0 * * * * /path/to/rake preheat:products RAILS_ENV=production 2>&1 >> #{Rails.root}/log/preheat.log &
Michael Koper
  • 9,586
  • 7
  • 45
  • 59
3

Probably the most basic solution would be to set up a simple cron entry to load up the page you'll want to have a 'hot' cache. This can be as easy adding the following to the crontab of a user on your server using crontab -e to open an editor:

*/15 * * * * wget -q http://yourwebpages.url/ > /dev/null 2>&1

What this will do is use wget to fetch the data at the provided url every 15 minutes of every hour, day, month and year, ignore the results and not send *nix mail in case something goes wrong.

Leftblank
  • 127
  • 1
  • 9
  • 1
    The controller is using Devise for authentication, so wget unfortunately would not be able to access the page without logging in. – gjb Mar 13 '11 at 00:45
  • 1
    You're right, that would be an issue. If you haven't found an alternative yet it could be an option to extend your app to support HTTP basic authentication, which Devise can do as well of course, that way it'd be easily possible to still use the basic wget-way. The only alternative I can think of would be to look at the caching code and attempt to trigger it with ruby code, but I'm not familiar enough with these functions to give that a shot. – Leftblank Mar 26 '11 at 14:55
  • An excellent suggestion, thank you. However, I decided to go with @bjpirt's answer and utilise curl's cookie support instead. – gjb Mar 30 '11 at 08:24