Apache is not sending 304 response (if mod_deflate and AddOutputFilterByType is enabled)

Question

I have added the following line in my Apache httpd.conf: -

AddOutputFilterByType DEFLATE text/html text/css application/javascript application/x-javascript application/json

I have a html file (test.html) with a script inclusion: -

<script type="text/javascript" src="/test.js"></script>

The problem is, every time I load test.html, test.js is also loaded with HTTP status: 200.

The question is: Why conditional GET is not satisfied?

If I comment out the "AddOutputFilterByType" line in httpd.conf, Apache sends 304.

If I enable AddOutputFilterByType in httpd.conf, the request header is: -

Host: optimize
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 GTB5 (.NET CLR 3.5.30729) FirePHP/0.2.4
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://optimize/
Cookie: PHPSESSID=nbq6h0eeahkshkcbc6ctu2j2b4
If-Modified-Since: Tue, 19 May 2009 07:06:46 GMT
If-None-Match: "2000000000717f-2c25a-46a3e8dcc2ad8"-gzip
Cache-Control: max-age=0

And the response header is: -

Date: Fri, 22 May 2009 07:03:40 GMT
Server: Apache/2.2.9 (Win32) PHP/5.2.6
Last-Modified: Tue, 19 May 2009 07:06:46 GMT
Etag: "2000000000717f-2c25a-46a3e8dcc2ad8"-gzip
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 52583
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: application/javascript

UPDATE: I have noticed, if I am disabling ETag, it works properly. I mean it sends 304.

FileETag None

But I really want to keep ETag as it is (I know that there is a inode disclosure issue).

Answer 1

This is a known bug in Apache. See Apache bug #45023, and summary of Apache 304 etags and mod_deflate.

Rebuilding from svn will fix the issue. The resolution was to revert the change that appended "-gzip" to the etag. However, there are associated HTTP compliance problems.

If you can't rebuild Apache, there is a suggested runtime configuration workaround in the bug report:

 RequestHeader  edit "If-None-Match" "^\"(.*)-gzip\"$" "\"$1\""
 Header  edit "ETag" "^\"(.*[^g][^z][^i][^p])\"$" "\"$1-gzip\""

Answer 2

"I've also decided that ETags aren't that useful in Apache anyway."

Wrong,
for example you have a file with modification date set to '2016.07.27 05:00:00', you upload it to your site, browser gets this file with HTTP code 200, then caches it and revalidates every time with HTTP 304.
Next you upload a file with the same filename again, but with older timestamp '2013.07.27 05:00:00' and with other content.

If ETag is disabled on server, browser will use only If-Modified-Since: request to determine if file was changed on server, so the request will be If-Modified-Since: 2016.07.27 05:00:00, but the file is not modified after this date, so a HTTP 304 is returned, even if the file has changed.

If ETag is enabled on server, besides If-Modified-Since:, there will be a If-None-Match: header coming from browser that will detect that file was changed(by default - timestamp mismatch+size mismatch) and the file will be redownloaded.

This problem still exists in Apache 2.4.23, so, I've written a better code than above to fix this issue. Expanation line by line:

1) If the browser sends a 'If-None-Match' request which has '-gzip' at the end, set variable request_etag=gzip.
2) Edit request header to strip out '-gzip' part.
3) Edit response header to add '-gzip' part, but only if the browser sent a '-gzip' request initially or response content is gzip encoded.

You can use either negative lookahead or negative lookbehind, regex speed is the same, Apache supports both

\"(.+(?<!-gzip))\"       #using negative lookbehind
\"((?:.(?!-gzip\"))+)\"  #using negative lookahead

Test cases:

"2e2-5388f9f70c580-afeg"
"2e2-5388f9f70c580-gzin"
"2e2-5388f9f70c580-gzipd"
"2e2-5388f9f70c580-gzip"
"2e2-5388f9f70c580gzip"

Copy-Paste this code into Apache .conf

SetEnvIf           If-None-Match "-gzip\"$" request_etag=gzip
RequestHeader edit If-None-Match "(.+)-gzip\"$" "$1\""
Header edit        ETag     "(.+(?<!-gzip))\"$" "$1-gzip\"" "expr=reqenv('request_etag') == 'gzip' || resp('Content-Encoding') == 'gzip'"

I personally use the following code, that strips '-gzip' part initially if it's a gzip response, and doesn't reappend it, so the browser will never send a '-gzip' 'If-None-Match' header.

Header edit ETag "(.+)-gzip\"$" "$1\"" "expr=resp('Content-Encoding') == 'gzip'"

Answer 3

I know this is a very old question, but it appears there's a more up-to-date answer.

To have Apache not append the -gzip suffix, you must use the DeflateAlterETag directive with a value of NoChange.

See the documentation for this here: http://httpd.apache.org/docs/trunk/mod/mod_deflate.html#deflatealteretag

Answer 4

Maybe you use a (squid) proxy which manipulates the HTTP Requests?