6

have stumbled upon something weird when enabling Rack::Deflater to gzip my response body. Perhaps i'm missing something but with this enabled the response is compressed however the ETag for the resource changes on every single request. This is forcing the app to respond each time rather then sending a 304. This works without Rack::Deflater enabled and I have verified that the page source is not changing. Am running a rails app with thin as the web server.

Gemfile.lock https://gist.github.com/2510816

Is there someway I can get a bit more output from rack middleware so I might be able to see whats going on?

Thanks in advance.

Nick Dancer
  • 335
  • 2
  • 9

1 Answers1

11

So I have fixed my original issue but still not quite getting the desired results. Turns out Rack::Deflater needs to be before Rack::ETag in the middleware stack. Still not sure why this would cause the ETag to change every request but sure enough if I change config.middleware.use "Rack::Deflater" to config.middleware.insert_before "Rack::ETag", "Rack::Deflater" then the ETag becomes consistent across requests. I'm still not getting a 304 but I think thats because of incorrect cache-control headers and unrelated to the original problem. Hopefully this helps someone in the future.

Nick Dancer
  • 335
  • 2
  • 9
  • FYI the `Rack::ETag` middle where is just creating the tag for whatever it is getting. Put it after `Rack::Deflater` in the middleware stack and it will attempt to create the hash of the compressed data, which is incorrect. – Michael Slade Apr 28 '12 at 12:43
  • Yeah that makes sense but even if it is generating it from the compressed data, if the input data and therefore compressed data have not changed then shouldn't the ETag that the server is generating still remain the same. – Nick Dancer Apr 28 '12 at 12:59
  • As long as the algorithm is deterministic and doesn't add any data from outside. You can't count on either. At a guess, the compression is adding a timestamp. Or maybe randomising its hash function. – Michael Slade Apr 28 '12 at 13:06
  • 7
    Found it. `GzipWriter`, which is what `Rack::Deflater` uses, adds a timestamp to everything it compresses. – Michael Slade Apr 28 '12 at 13:19
  • Ok that makes a lot more sense then. Thanks for your explanation. – Nick Dancer Apr 28 '12 at 14:08
  • Actually an ETag based on the compressed data *is* correct. Response bodies for the same ETag should be byte-for-byte comparable. Otherwise a weak ETag validator should be used e.g 'ETag: W/"#{md5(non-gzip-body())}"'. – mlangenberg Jun 15 '13 at 14:29