Why AWS does not trust to pre-signed URLs generated by its own SDK

Question

I'm composing pre-signed URL for downloading objects from AWS S3. I use generatePresignedUrl method from AWS Java SDK. However, I get SignatureDoesNotMatch from AWS when making GET request using the generated pre-signed URL. I'm confused cause I use a method from the official SDK to generate it and a very simple GET request, but no luck. Any help much appreciated!

The code (taken from AWS docs):

java.util.Date expiration = new java.util.Date();
long milliSeconds = expiration.getTime();
milliSeconds += 1000 * 60 * 60; // Add 1 hour.
expiration.setTime(milliSeconds);

GeneratePresignedUrlRequest generatePresignedUrlRequest = 
new GeneratePresignedUrlRequest(bucketName, objectKey);
generatePresignedUrlRequest.setMethod(HttpMethod.GET); 
generatePresignedUrlRequest.setExpiration(expiration);

URL url = s3client.generatePresignedUrl(generatePresignedUrlRequest); 

Credentials which I use for connecting to AWS S3 are tested; I was able to download S3 objects using same credentials and S3 download method from AWS SDK. But the response I get from AWS to the composed URL (for example, https://ozland.s3.amazonaws.com/1865b563cdc94fa28ef41ee0b9b0e608?AWSAccessKeyId=...AWSAccessKeyId...&Expires=1471300223&Signature=pbgcRB0Zg%2B3iicDQQbVX%2FqdNAAc%3D) from above is:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message
<AWSAccessKeyId>...AWSAccessKeyId...</AWSAccessKeyId>
<StringToSign>GET


1471300223
/ozland/1865b563cdc94fa28ef41ee0b9b0e608</StringToSign>
<SignatureProvided>pbgcRB0Zg+3iicDQQbVX/qdNAAc=</SignatureProvided>
<StringToSignBytes>47 45 54 0a 0a 0a 31 34 37 31 33 30 30 32 32 33 0a 2f 61 6c 61 62 61 6d 61 63 6f 75 6e 74 79 2f 31 38 36 35 62 35 36 33 63 64 65 39 34 66 61 32 38 65 66 34 31 65 65 30 63 39 62 30 65 36 30 38</StringToSignBytes>
<RequestId>...RequestId...</RequestId>
<HostId>...HostId...</HostId></Error>

The question is why response.StringToSign from above differs from the way AWS docs (AWS docs) describe it. In particular, according to the docs, I expect StringToSign to be like:

AWS4-HMAC-SHA256
20150830T123600Z
20150830/us-east-1/iam/aws4_request
f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59

aws-tools works just fine for me for generating pre-signed URLs using exact same S3 bucket and credentials. But why am I getting SignatureDoesNotMatch here anyway?

Thank you!

Answer 1

The difference is that the string-to sign in the error response is in the Signature Version 2 format, and this in turn is because your signed URL is in V2 format.

Among the notable visual differences between the two formats, V2 has Signature=[base64] while V4 has X-Amz-Signature=[hex].

This explains the difference in the strings-to-sign.

It doesn't get you any closer to explaining why the signature does not match, because V2 is perfectly valid in regions launched before 2014.

The most likely culprit is usually a missing character or copy/paste error in the secret, or whatever user agent you're using to make the request is doing some funny business. If the string to sign matches the request you're making -- and it looks like it does -- there's not much that can go wrong in V2 if your keys are named using characters in the subset of ASCII that doesn't require urlencoding -- as this request appears to be. (Valid characters in keys are UTF-8, but ASCII alphanumeric is pretty well universally unmangled by all user agents).