More specific rules for an AWS Canonical Request header list?

Question

The AWS documentation here seems to have somewhat confusing, incomplete or contradictory information. It states that

CanonicalHeaders is a list of request headers with their values.

Which suggests that we'd put all request headers in the canonical request. However, later, they state

The CanonicalHeaders list must include the following:

HTTP host header

If the Content-Type header is present in the request, it must be added to the CanonicalHeaders list.

Any x-amz-* headers that you plan to include in your request must also be added. For example, if you are using temporary security credentials, you will include x-amz-security-token in your request. You must add this header in the list of CanonicalHeaders.

OK, the bit about the Content-Type and x-amz headers suggests that we don't actually take all headers, because otherwise they wouldn't need to state that they'd be must be included. So then perhaps, we only need to take the Host header, the Content-Type header, and any x-amz-* headers. But then below, it gets more confusing, because here's an example request:

GET /test.txt HTTP/1.1
Host: examplebucket.s3.amazonaws.com
Date: Fri, 24 May 2013 00:00:00 GMT
Authorization: SignatureToBeCalculated
Range: bytes=0-9 
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20130524T000000Z 

And here's the example canonical request created from it:

GET
/test.txt

host:examplebucket.s3.amazonaws.com
range:bytes=0-9
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20130524T000000Z

host;range;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 

But this is inconsistent with both of the interpretations earlier: if we're supposed to only have Content-Type, Host and x-amz-* headers, then what is the range header doing in the list? And if we're just supposed to take all of the headers, then why isn't the Date header in the list?

Is the list of headers to put in a canonical request then arbitrary, as long as it contains at least the minimum headers? What, exactly, is the definitive set of rules to construct the canonical request headers?

Answer 1

if we're supposed to only have Content-Type, Host and x-amz-* headers, then what is the range header doing in the list?

You are only required to have Content-Type, Host, and x-amz-*, but you can add other headers that you would like to add to the signature to be validated.

See the note in the docs that says: "For the purpose of calculating a signature, only the host and any x-amz-* headers are required; however, in order to prevent data tampering, you should consider including all the headers in the signature calculation."

And if we're just supposed to take all of the headers, then why isn't the Date header in the list?

The Date header is special, because it is added by the browser according to client system time that may be incorrect. Because of that, you can use x-amz-date instead.

Is the list of headers to put in a canonical request then arbitrary, as long as it contains at least the minimum headers?

Yes!

What, exactly, is the definitive set of rules to construct the canonical request headers?

That would be those defined in the documentation of AWS signature version 4... but you got the idea: you must sign the minimal set of request data and can sign all headers that you'd like.

That said, avoid all this if you can. The SDK for [Javascript, Java, .NET, Python, Ruby, PHP,...] already sign requests for you, manages temporary credentials, credential chains, threading, retries and a lot more. If you can use that, it would probably save a lot of headache.