0

I am downloading a Helm chart from https://kubernetes-charts.storage.googleapis.com/redis-0.5.1.tgz. (The fact that it is Redis or related to Helm or anything in particular is irrelevant to this question, which is just about things like Content-Encoding and so on.)

When I check its headers like this:

$ curl -H "Accept-Encoding: gzip" -I https://kubernetes-charts.storage.googleapis.com/redis-0.5.1.tgz

…I do not see a Content-Encoding header in the output, and the Content-Type is listed as being application/x-tar:

HTTP/1.1 200 OK
X-GUploader-UploadID: AEnB2UqBzSXfTToMAdMARXSjJeN0on3jaNY3u74eXcWfvqsOwRpi38Xc6T0XrrmY4otPeySaYRwXyHccHYtChoPAgFQwYZhQMhcpZRWtZURRANGdfRJoupI
Expires: Tue, 27 Jun 2017 00:21:59 GMT
Date: Mon, 26 Jun 2017 23:21:59 GMT
Cache-Control: public, max-age=3600
Last-Modified: Fri, 05 May 2017 03:03:41 GMT
ETag: "e4184c81a58fb731283847222a1f4005"
x-goog-generation: 1493953421241613
x-goog-metageneration: 1
x-goog-stored-content-encoding: identity
x-goog-stored-content-length: 3550
x-goog-meta-goog-reserved-file-mtime: 1493953414
Content-Type: application/x-tar
x-goog-hash: crc32c=bQHveg==
x-goog-hash: md5=5BhMgaWPtzEoOEciKh9ABQ==
x-goog-storage-class: STANDARD
Accept-Ranges: bytes
Content-Length: 3550
Server: UploadServer
Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,36,35"

The resulting file, when downloaded, is a gzipped tar archive.

What is the proper way of programmatically detecting that the payload is in fact gzipped? Or is this a problem with the web server in question?

Laird Nelson
  • 15,321
  • 19
  • 73
  • 127

2 Answers2

0

I think the server is misconfigured. Since .tgz is just abbreviation for .tar.gz it should get the content type application/gzip-

fhossfel
  • 2,041
  • 16
  • 24
  • Thank you for your answer. That's my understanding as well, although I see from @N3SS4H's answer that there is no fundamentally agreed-upon MIME type for a gzipped tar file. – Laird Nelson Jun 26 '17 at 23:47
  • The Helm folks seem to think it is misconfigured as well: https://github.com/kubernetes/charts/issues/1383#issuecomment-311310420 Interestingly, though, if the object-at-rest is a gzipped file, then `Content-Encoding` seems like the wrong way to solve this problem. In any event, your suspicion is correct so I'm accepting your answer. Thanks! – Laird Nelson Jun 27 '17 at 16:25
0
Content-Type: application/x-tar

this header tells you the type, but i'm not sure that's gzip

https://superuser.com/questions/901962/what-is-the-correct-mime-type-for-a-tar-gz-file

see accepted answer at How to check if a file is gzip compressed? . for way to identify programmatically

N3SS4H
  • 561
  • 4
  • 11
  • Thanks for your answer. Indeed, the `Content-Type` header is telling me that the underlying object at rest is a `.tar` file. (Maybe that's true (propertly configured); maybe it's not.) _If_ that's true, then since I am receiving a stream of gzipped bytes, and not a streamed `.tar` file, then I have to have a way to see that `gzip` compression was applied to the object _in flight_. I see no such mechanism here. – Laird Nelson Jun 26 '17 at 23:46
  • https://stackoverflow.com/questions/6059302/how-to-check-if-a-file-is-gzip-compressed – N3SS4H Jun 26 '17 at 23:57
  • 1
    Thanks; yes; you are right that I can sniff the first two bytes of the stream to see if they match the identifiers of a `gzip`ped file but that can yield false positives. My concern here is not simply "I need this to work now" (I can make it work with all sorts of various workarounds) but more like: "Am I missing something?" – Laird Nelson Jun 27 '17 at 00:10