Why does HttpWebResponse try to decompress the stream using GZip when Content-Encoding is empty?

Question

I've looked at a lot of Q&As around this subject and got to the point where I use the following code in order to get the bytes from a given URI:

var request = (HttpWebRequest)WebRequest.Create(uri);
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
var response = request.GetResponse();
var stream = response.GetResponseStream();
if (stream != null)
{
    var buffer = new byte[4097];
    var memoryStream = new MemoryStream();

    do
    {
        var count = stream.Read(buffer, 0, buffer.Length);
        memoryStream.Write(buffer, 0, count);

        if (count == 0)
            break;
    } while (true);

    return memoryStream.ToArray();
}

response.Close();

return null;

Now, for a certain URI (which is pointing to a file), when debugging, I see that the 'Content-Encoding' header of the web response equals to nothing (""), but when trying to read from the stream, it throws an exception:

System.IO.InvalidDataException: The magic number in GZip header is not correct. Make sure you are passing in a GZip stream.

When debugging the same URI in dev tools I get this on the response headers:

Content-Encoding:gzip,deflate

So I really don't know what happens.

Any clues and ideas on how to avoid this exception and successfully read the file's bytes?

Thanks!

score 0 · Answer 1 · answered Mar 31 '17 at 22:07

The .NET framework blanks this header out as part of the auto-decompression. I would assume this is because you are asking the framework to handle it automatically and that the response stream you get back is no longer compressed.

I had to just look this up myself by inspecting the source code HttpWebResponse at Microsoft

Why does HttpWebResponse try to decompress the stream using GZip when Content-Encoding is empty?

1 Answers1