8

I have a web page that returns the following header when I access material:

HTTP/1.1 200 OK
Date: Sat, 29 Jun 2013 15:57:25 GMT
Server: Apache
Content-Length: 2247515
Cache-Control: no-cache, no-store, must-revalidate, max-age=-1
Pragma: no-cache, no-store
Expires: -1
Connection: close

Using a chrome extension, I want to modify this response header so that the material is actually cached instead of wasting bandwidth.

I have the following sample code:

chrome.webRequest.onHeadersReceived.addListener(function(details) 
    {
        // Delete the required elements
        removeHeader(details.responseHeaders, 'pragma');
        removeHeader(details.responseHeaders, 'expires');

        // Modify cache-control
        updateHeader(details.responseHeaders, 'cache-control', 'max-age=3600;')

        console.log(details.url);
        console.log(details.responseHeaders);

        return{responseHeaders: details.responseHeaders};
    },
    {urls: ["<all_urls>"]}, ['blocking', 'responseHeaders']
);

Which correctly modifies the header to something like this (based on the console.log() output):

HTTP/1.1 200 OK
Date: Sat, 29 Jun 2013 15:57:25 GMT
Server: Apache
Content-Length: 2247515
Cache-Control: max-age=3600
Connection: close

But based on everything I have tried to check this, I cannot see any evidence whatsoever that this has actually happened:

  1. The cache does not contain an entry for this file
  2. The Network tab in the Developer Console shows no change at all to the HTTP response (I have tried changing it to even trivial modifications just for the sake of ensuring that its not a error, but still no change).

The only real hints I can find are this question which suggests that my approach still works and this paragraph on the webRequest API documentation which suggests that this won't work (but doesn't explain why I can't get any changes whatsoever):

Note that the web request API presents an abstraction of the network stack to the extension. Internally, one URL request can be split into several HTTP requests (for example to fetch individual byte ranges from a large file) or can be handled by the network stack without communicating with the network. For this reason, the API does not provide the final HTTP headers that are sent to the network. For example, all headers that are related to caching are invisible to the extension.

Nothing is working whatsoever (I can't modify the HTTP response header at all) so I think that's my first concern.

Any suggestions at where I could be going wrong or how to go about finding what is going wrong here?

If its not possible, are there any other ways to achieve what I am trying to achieve?

Community
  • 1
  • 1
btalb
  • 6,937
  • 11
  • 36
  • 44

1 Answers1

5

I have recently spent some hours on trying to get a file cached, and discovered that the chrome.webRequest and chrome.declarativeWebRequest APIs cannot force resources to be cached. In no way.

The Cache-Control (and other) response headers can be changed, but it will only be visible in the getResponseHeader method. Not in the caching behaviour.

Rob W
  • 341,306
  • 83
  • 791
  • 678
  • Unfortunately what I was expecting... Do you know of any other ways to force these files to cache? I have found proxy servers like `squid` but they seem like a lot of fiddling with the network on my system just to cache files from one specific site – btalb Jun 29 '13 at 16:23
  • @BT Choose your favourite proxy which adds the right headers, and use the [`chrome.proxy`](https://developer.chrome.com/extensions/proxy.html) API to route that specific site through your proxy. – Rob W Jun 29 '13 at 16:25
  • Real shame that you can't modify caching from an extension. Would make it a lot more helpful if you could. – Franz Payer Jul 18 '13 at 00:52
  • If anyone else is working towards a solution for this problem, I got there in a very roundabout way. Like suggested, I used a Chrome extension to get any useful information from the page itself and a file io NPAPI plugin to save this output. Then I created my own caching proxy server through Twisted (easiest proxy implementation I could find) and used a bash script to process the cache and information obtained through the extension. VERY messy to say the least.... agree completely with @FranzPayer – btalb Aug 02 '13 at 02:49
  • @BT Is NPAPI really necessary? You could route all traffic through a local proxy server (that modifies headers) using the [`chrome.proxy`](https://developer.chrome.com/extensions/proxy.html) API. – Rob W Aug 02 '13 at 07:32
  • Problem was I needed to extract elements of the loaded DOM as well. Sorry, old question and just realised I didn't include that info in the question ;) – btalb Aug 03 '13 at 01:16
  • what about ssl connections? AFAIK you can't use a proxy to change the header values of those without triggering an ssl warning in the browser. – Jürgen Paul Apr 02 '14 at 09:32
  • Unless you install a root certificate for MITM, then it's not possible to modify these headers. I've investigated whether it's possible to support recognizing the cache headers from the webRequest API, but it turned out that it's quite difficult/not feasable. The caching directives are processed at the network transaction layer, before the request is delegated to the extension system (see also http://crbug.com/355232). – Rob W Apr 02 '14 at 09:53