0

Possibly related to Go: Detect gzip encoding to manually decompress response, but 'Content-Encoding' header missing

I understand that net/http Transport will add Accept-Encoding: gzip to a request unless DisableCompression is set to True, and that is required if I want it to automatically decompress a gzipped response. That being the case, the following code does not receive a Content-Encoding: gzip header:
https://play.golang.org/p/FWs5uG9pZEL (note: will not run in the playground due to network constraints)

If I run a local server and run the above code with it I can see the expected header being sent:

GET / HTTP/1.1
Host: localhost:5555
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip

Is there something else about the connection go creates that is causing the server not to return a gzipped response?

jonhadfield
  • 119
  • 1
  • 1
  • 9

1 Answers1

2

Accept-Encoding: gzip only means that the client is able and willing to handle gzip compressed content. It does not mean that the server must actually compress the content. In fact, for example with images it would not make any sense to compress the content with gzip because these are already compressed data (but not with gzip) and adding gzip on top might actually increase the size of the payload.

If you want your server to return compressed content you actually have to configure your server to do this. See for example here on how to do this with nginx.

Note that http.Response will transparently decompress the response and update the header accordingly, i.e. remove Content-Encoding. This means you will not get the original response header when checking with resp.Header.Get("Content-Encoding"). If the response was automatically decompressed can be seen in the Uncompressed field:

fmt.Println("was compressed ", resp.Uncompressed)

For more see go doc http.Response:

// Uncompressed reports whether the response was sent compressed but
// was decompressed by the http package. When true, reading from
// Body yields the uncompressed content instead of the compressed
// content actually set from the server, ContentLength is set to -1,
// and the "Content-Length" and "Content-Encoding" fields are deleted
// from the responseHeader. To get the original response from
// the server, set Transport.DisableCompression to true.
Uncompressed bool
Steffen Ullrich
  • 114,247
  • 10
  • 131
  • 172
  • I understand and agree with what you have put but my code example includes a request to https://golang.org which is a platform that does return a gzipped response if the client provides ```Accept-Encoding: gzip``` in the request. My question refers to the issue that go _appears_ to be sending the correct header yet I don't get the gzipped response and header. – jonhadfield Dec 01 '19 at 13:33
  • 1
    @jonhadfield: That `https://golang.org` will usually return compressed content is a detail I did no see in your question. I've updated the answer to add additional information, especially that go will automatically decompress the response and update the response header so you will not get the original `Content-Encoding` send by the server. – Steffen Ullrich Dec 01 '19 at 13:54
  • Thanks Steffen. I incorrectly assumed the response was returned uncompressed due to the absence of the Content-Encoding header. – jonhadfield Dec 01 '19 at 14:39