10

We have 8 REST-ish API servers with Nginx using FastCGI with PHP-FPM to handle requests. We're currently using Nginx' FastCGI caching (directives like fastcgi_cache_path). This means that API responses are cached, but there is a separate cache for each server.

Is there a good way to share cache storage among all eight servers?

We have considered using Redis as shared storage, but the modules available seem to require application changes. In some cases, we may wish to cache responses outside of our control (over HTTP to external APIs). Ideally, a drop-in replacement for the Nginx built-in caching of FastCGI and HTTP responses would be available.

Brad
  • 1,419
  • 22
  • 43
  • Why don't you make caching not on frontend layer (Nginx), but on backend layer (PHP application)? – Max Kochubey Mar 26 '14 at 03:42
  • 1
    @hangover I see no reason to re-invent the wheel. Nginx cache works well and fast. If we can avoid the overhead of running requests through the application at all, we can keep things nice and fast. Nginx responds to cached requests for us in under 5 milliseconds. Launching our PHP application, even for it to provide a cached response, will probably 10x slower than that. Nginx caching is working well for us now, we just need to distribute that cache among many servers. – Brad Mar 27 '14 at 19:12
  • 1
    ok, than you can try to use 3rd-party Nginx [SRCache](http://wiki.nginx.org/HttpSRCacheModule) module to store cached content in dedicated Memcached od Redis. – Max Kochubey Mar 28 '14 at 08:25
  • 1
    @Brad This is hacky so I won't put it as an answer: we currently have a nightly run script that uses rsync between the nginx cache directories on our servers to keep each up to date with the latest cache files from all the others. Script ends with a graceful nginx restart and success check. Like I said, it's hacky, but it works for us on a high throughput system. – mVChr Jul 24 '14 at 16:34
  • @mVChr Interesting. Have you tried running rsync or similar continuously? In the end for my use, I eventually got Redis working but this doesn't allow for a very large cache since the entire Redis collection must be in memory. 16GB fills up fast on my application. – Brad Jul 24 '14 at 16:36
  • @Brad we haven't tried running rsync continuously because 1) like I said this is a very high throughput and integral system (think part of the back end for multiple Alexa top 500 site(s)) so we want to be able to monitor and isolate specific procedures in case of issues 2) we are in the process of building a new version of this system that handles the caching issue differently, so we're loathe to mess with the old system too much since it's working and will be replaced (hopefully) soon – mVChr Jul 24 '14 at 18:04

1 Answers1

1

There seems to be a rather new blog post at https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/ about this issue. This first example could be usefull if you run more than two nginx cache servers.

Especially the second part of the post sounds interesting for my current use-case, where I want to automatically re-crawl changed items into my caches.

It should work with the open source Version of NGINX too. Basically it works by proxying the request cascaded through each NGINX Server (Nginx-Cache1->NGinx-Cache2->Origin-Server) each server caches from the relevant upstream and if desired it is possible to create a HA Cluster too. https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/

macbert
  • 153
  • 1
  • 9