4

While running Fiddler, I noticed something odd in a request for a static ~5MB XML file to my server: Despite sending byte-for-byte identical headers (Edit: Including the headers), the responses were different:

Response A:
1. 700KB of gzip content
2. Included the Content-Length header
3. Excluded Transfer-Encoding header

Response B:
1. 1000KB of gzip content
2. Excluded the Content-Length header
3. Included Transfer-Encoding header: chunked header

What can I do so that I consistently receive the more bandwidth-efficient behavior shown in response A?

Raw Request:

GET http://[REDACTED]/[REDACTED]/[REDACTED]/[REDACTED].xml?dt=Test1 HTTP/1.1
Host: [REDACTED]
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

Raw Response A:

HTTP/1.1 200 OK
Content-Type: text/xml
Content-Encoding: gzip
Last-Modified: Tue, 07 May 2013 04:04:01 GMT
Accept-Ranges: bytes
ETag: "80ceefe7d74ace1:0"
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Tue, 07 May 2013 21:07:21 GMT
Content-Length: 728105

[700KB GZipped Body]

Raw Response B:

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: text/xml
Content-Encoding: gzip
Last-Modified: Tue, 07 May 2013 04:04:01 GMT
Accept-Ranges: bytes
ETag: "60be30e8d74ace1:0"
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Tue, 07 May 2013 21:07:14 GMT

[1MB Gzipped Body]
Brian
  • 303
  • 1
  • 3
  • 15

3 Answers3

6

The compression of static files is handled dynamically (if dynamic compression is enabled) while the file is considered infrequent by IIS. Once the file is considered frequent it will be compressed and cached (if static compression is enabled). The cached version will continue to be served until it becomes infrequent again. There are 2 config settings you can use in IIS to configure frequent files:

system.webServer/serverRuntime:

  • frequentHitThreshold: How many times should the same file be requested, before it's considered frequent and cached? Defaults to 2.
  • frequentHitTimePeriod: Time interval in which the same file should be requested {frequentHitThreshold} times, in order to be cached. Defaults to 10 seconds.

Beware that regardless of the frequentHitTimePeriod you set, a frequent file will always become infrequent if it is not requested after 1 minute. I have no idea if there is a setting for this in the config.

Setting frequentHitThreshold to 1, for example, will mean that the file is always considered frequent by IIS, even from the first request. This will in turn bypass the dynamic compression and be treated only by static compression.

Note that the compression levels for dynamic (default 0) and static (default 7) compression are different so will return 2 different file sizes.

Also and this is why I got into this issue in the first place: the ETag for the same file is different between dynamic and static compression even if you use the same levels for both.

Hope this helps.

ianbeks
  • 193
  • 2
  • 5
2

Apparently, on the first request for a static file, IIS does not have a compressed copy of the file in it's compressed file cache, so it uses dynamic compression on that request. This can be resolved by setting the serverRuntime element's frequentHitTHreshold attribute to 1.

This is discussed in detail here. This setting is probably only worth changing if serving a CDN.

Brian
  • 303
  • 1
  • 3
  • 15
0

Look here for more details on IIS dynamic compression:

http://www.west-wind.com/weblog/posts/2011/May/05/Builtin-GZipDeflate-Compression-on-IIS-7x

Basically with high CPU load on the server, less compression is done

Here more details and how to configure the compression levels:

http://weblogs.asp.net/owscott/archive/2009/02/22/iis-7-compression-good-bad-how-much.aspx

André Schild
  • 258
  • 3
  • 9
  • This is a static file, so the dynamic compression rules should be inapplicable. I do not believe that static files are more weakly compressed under heavy load. Further, the CPU usage at the time of the two requests was roughly equal (20%), so I doubt that is a factor. – Brian May 07 '13 at 21:22
  • If static compression is done, then it should return the same thing. (You should be able to see the compressed file in the corresponding system folder) What is strange, is that answer B shows "Transfer-Encoding: chunked" which is usually only sent when the server does not yet know the response size (Means dynamic in some way) Are there perhaps some other output filters in effect ? – André Schild May 07 '13 at 21:46