nginx/memcached for cached web proxy

Question

I'm trying to setup nginx as a caching layer on top of an HTTP proxy - along with memcached to do the actual caching. The goal here is that we have an internal application which will make frequent outgoing HTTP requests, and we want to use nginx+memcached as a web proxy to speed up the responses to these outgoing client requests. (Especially since many requests may be redundant).

There's a lot of nice documentation and information on setting up nginx along with memcached out there, but I'm running into a problem because it seems like my particular use case isn't too common. Most tutorials, examples, etc. assume the reader is running a website, and wants to use nginx+memcached to increase website performance.

But my use-case is different. I just want to use nginx as a web caching proxy, and use memcached as the actual storage layer for caching. Again, I'm not interested in speeding up performance for incoming requests to a website - I'm interested in speeding up outgoing HTTP requests from an internal app.

So, although there seems to be a lot of tutorials online which explain how to set up nginx as a web cache proxy, as well as many tutorials online which explain how to setup nginx+memcached as a high-performance caching webserver, there doesn't seem to be much in the way of explaining how to setup nginx as a web cache proxy that uses memcached to do the actual caching.

Okay, so I've tried to read between the lines and figure it out myself. I have nginx and memcached both installed locally on a dev box. I'm also using the "Enhanced Nginx Memcached Module", rather than the nginx+memcached module that comes with nginx out of the box, because apparently the out-of-the-box version doesn't cache HTTP headers (which would seem to make it unusable in practice, unless I'm missing something here).

So the relevant portion of my nginx.conf file is:

upstream some-upstream-proxy {
    server proxy.giant-corporation.com:82;
}

upstream memcached_upstream {
    server 127.0.0.1:11211;
    keepalive 20;
}

server {
    listen       80 default_server;
    server_name my-cache-proxy;

   location @fallback {
        proxy_pass http://some-upstream-proxy;
        proxy_cache my_cache;
        proxy_redirect default;
        proxy_set_header Host      $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location / {
        set $enhanced_memcached_key "$request_uri";
        set $enhanced_memcached_key_namespace "$host";
        set $enhanced_memcached_use_add 1;
        set $enhanced_memcached_expire $http_memcached_expire;
        enhanced_memcached_hash_keys_with_md5 on;
        enhanced_memcached_pass memcached_upstream;

        error_page 404 = @fallback;
    }
}

So, my understanding here is that what this is supposed to do is the following:

1. An HTTP request is made to nginx.
2. Nginx looks up the URI in memcached. If it finds the URI, it returns the cached response data.
3. If it doesn't find the URI, it falls back on @fallback and uses the upstream HTTP proxy to make an HTTP request.
4. It then caches that request in memcached and returns the response to the client.

So my nginx.conf file seems to be mostly doing what I want, except for the last step. It looks like it's adding in an entry for the URI, but the actual cached value appears to be 0 bytes.

For example, if I run nginx and memcached with the above nginx.conf file, and then I say:

 curl http://www.google.com/ --proxy http://localhost:80

... I get the actual response data from Google. Then, if I check memcached stats, I see:

STAT cmd_get 21
STAT cmd_set 1

And after each request I see cmd_get is increasing. But if I do an actual memcached dump, I see:

$ memcached-tool localhost:11211 dump
Dumping memcache contents
  Number of buckets: 1
  Number of items  : 1
Dumping bucket 1 - 1 total items
add __ns__www.google.com 0 1479132450 1
0

So I see that nginx is definitely communicating with memcached, and has successfully inserted the URI I requested. But it doesn't appear to be actually caching the response it gets from the http proxy server. It just has a 0 byte entry for www.google.com.

So... what step am I missing here? How can I modify my nginx.conf file to do what I want?