3

I have been working on getting gzip/deflate compression working on Web API responses. I have been using the code from Github - MessageHandlers.Compression. However it didn't appear to work. There was no Content-Encoding header appearing in the Google Developer console or in Firebug in Firefox and the Content-Length was consistently set to the uncompressed size of the data. So I kept stripping out the code until I ended up with the following:

protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
    // Send the request to the web api controller
    var response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);

    // Compress uncompressed responses from the server
    if (response.Content != null && request.Headers.AcceptEncoding.IsNotNullOrEmpty())
    {
        var content = response.Content;
        var bytes = content.ReadAsByteArrayAsync().Result;

        if (bytes != null && bytes.Length > 1024)
        {
            // The data has already been serialised to JSon by this point
            var compressedBytes = Compress(bytes);
            response.Content = new ByteArrayContent(compressedBytes);

            var headers = response.Content.Headers;
            headers.Remove("Content-Type");
            headers.ContentLength = compressedBytes.Length;
            headers.ContentEncoding.Clear();
            headers.ContentEncoding.Add("gzip");
            headers.Add("Content-Type", "application/json");
        }
    }

    return response;
}

private static byte[] Compress(byte[] input)
{
    using (var compressStream = new MemoryStream())
    {
        using (var compressor = new GZipStream(compressStream, CompressionMode.Compress))
        {
            compressor.Write(input, 0, input.Length);
            compressor.Close();
            return compressStream.ToArray();
        }
    }
}

When I initially did this I made a mistake, and set the content encoding in the header to 'gzip' when I used a DeflateStream in the Compress method. As you would expect I got an error in the browser, however the response headers were correct(!). That is, the Content-Encoding header was set and the Content-Length was correct. As well, looking at the raw data I could clearly see if was compressed. As soon as I corrected my error though the problem returned.

What I am wondering is do the latest versions of the browser decompress the content behind the scenes, or is there actually something wrong with my code? Responses are sent in Json format

Any help much appreciated.

EDIT

I tried dumping the headers to a log file in the following methods in Global.asax (listed in the order they appeared in the log):

Application_PreSendRequestHeaders

Application_EndRequest

Application_PreSendRequestContent

In each case the required headers were there, even though they didn't appear in the Google developer console. I then took a look at the solution at Code Project. When run from the command line everything worked as anticipated. However when I called the web server from Google Chrome I got the exact same result. That is, no Content-Encoding header, and no indication as to whether the content had been compressed or not. However with the developer console open it's easy to see this header at other sites (stack overflow for instance). I have to assume this is therefore something to do with compressed responses from web api services. It's so hard to know though if this is actually working client side.

fhevol
  • 934
  • 10
  • 28
  • Did you ever get any of this to work correctly from within the ASP .Net Web API framework? So far I have not had *any* luck at all returning a gzip'ed JSON response -- the Content-Encoding response header value is always removed by this worthless API, and no matter what I try, the header is always stripped out. Just one more example of how horribly broken this API is. If I add something to the response header, then **LEAVE IT THE HELL ALONE!**. – jerhewet Feb 26 '15 at 18:05
  • Another thing. Please, no pointers to other posts here on SO, or posts from five years ago on the internet. None of them work. All of them end up with the Content-Encoding header being stripped out of the response. All of them. – jerhewet Feb 26 '15 at 18:07
  • Unfortunately no, I never got beyond what you see here. Due to time constraints I simply had to move on. – fhevol Mar 04 '15 at 01:47
  • Actually I don't think just the header is being removed, I think the data is being decompressed by the API as well. I could well be wrong but I have 2 reasons for thinking this. Firstly, the size of the download as displayed in the developer console in Chrome exactly matches that when the compression logic isn't running. Secondly, when I set the header to gzip, but used Deflate to compress the data the headers appeared in Chrome, but an error occurred as the data couldn't be decompressed. The headers still appearing suggests the API was also unable to decompress the data, so left it alone. – fhevol Mar 05 '15 at 22:12
  • 1
    Looks like [it's something stupid](http://stackoverflow.com/questions/13516844/) that Microsoft has done. So it's highly likely that it's nothing we're doing -- it's the result of Microsoft's new "security by obscurity" policies. – jerhewet Mar 06 '15 at 15:23
  • Thanks for letting me know about this, much appreciated. So this is happening client side then(?). – fhevol Mar 17 '15 at 20:03
  • The problem is caused by Windows Defender, so you should try running Programs and Features > Repair on your existing anti-virus software to see if that fixes it, or a complete uninstall / reinstall of your A/V if it doesn't. In my case, I took ownership of everything related to Windows Defender and deleted it, and the problem vanished. – jerhewet Mar 18 '15 at 12:04
  • Thank you so much for your help. I would never have thought to look at the anti virus software. We run Security Manager AV Defender here. Once I have found out how to turn this thing off I will test and report back. Huge thanks once again! – fhevol Mar 18 '15 at 21:24
  • Once the anti virus was switched off (by the IT department) everything worked as anticipated. As this occurs client side I actually don't mind it happening - my data is still compressed when transmitted over the wire. The biggest thanks once again, it would never have occurred to me in a month of Sundays to look at the anti-virus app. – fhevol Mar 19 '15 at 01:45

1 Answers1

3

In case anyone doesn't want to read all the comments the answer came from Jerry Hewett (jerhewet). Namely that anti-virus software intercepts the response before it gets to the browser. The anti-virus software decompresses the data, no doubt as part of the scanning process. Huge thanks to Jerry for his help here.

fhevol
  • 934
  • 10
  • 28