How do I utilize AWS Signature v4 when generating a presigned S3 URL?

Question

I'm inheriting a codebase that makes use of the Java AWS SDK to generate presigned S3 URLs for both Putting and Getting Objects. The code looks something like this:

GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucket, filename);
request.setMethod(HttpMethod.PUT);
request.setExpiration(new DateTime().plusMinutes(30).toDate());
request.setContentType("image/jpeg");
String url = awss3.generatePresignedUrl(request);

And this existing codebase has always worked, and is very close to working. However, one business requirement that changed is that we need to encrypt the contents of the S3 bucket. So, naturally, I set the default encryption on the bucket to AWS-KMS (since it seemed like the most modern) and choose the default "aws/s3" key that had been created for my account.

However, now when an end user tries to actually utilize the URLs I generate in their browser, this is the error message that appears:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <code>InvalidArgument</code>
  <Message>Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.</Message>
  <ArgumentName>Authorization</ArgumentName>
  <ArgumentValue>null</ArgumentValue>
  <RequestId>...</RequestId>
  <HostId>...</HostId>
</Error>

My question is utlimately: how do I get this working again? As I see it there are two different paths I could take. Either: 1) I could downgrade the bucket encryption from AWS-KMS to AES-256 and hope that it all works, or 2) I can make some change to my client code to support KMS, which I'm guessing would probably involve downloading the KMS key through the AWS SDK and using it to sign the requests, and possibly also adding some Authorization and other headers.

Option 1 seems like less work but also less ideal, because who knows if a less secure form of encryption will always be supported. And Option 2 seems like the better choice conceptually, but also raises some concerns because it does seem like a lot more work and I'm worried about having to include extra headers. The code I've shown above reflects the equivalent of a PutObject request (proxied through the generated URL), but there are also equivalents of GetObject requests to download the images, which are possibly rendered directly in the browser. It would be a lot harder to write frontend code there to use different headers just to render an image. (I wonder if query parameters can be substituted for headers?)

Anyways, what would I need to change in my Java to get this working with AWS KMS? Do I need to use the AWS SDK to "download" the KMS key first as I suspected? And should I go about doing it that way, or would AES-256 really be the better option?

Are you using ClientConfiguration().withSignerOverride("AWSS3V4SignerType")). Then add .withSSEAlgorithm(SSEAlgorithm.KMS.getAlgorithm()) when calling GeneratePresignedUrlRequest. You can also specify the KMS Key ID with .withKmsCmkId(id) https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ServerSideEncryptionByDefault.html — John Hanley, Apr 29 '18 at 22:11

score 0 · Answer 1 · answered Apr 29 '18 at 22:20

0

Signing signature verion 4 has been the default for several years. Unless you are overriding the signature in your AWS SDK profile, then you are using version 4. You can override this using the following code:

AmazonS3Client s3 = new AmazonS3Client(new  ClientConfiguration().withSignerOverride("AWSS3V4SignerType"));

Most likely the real issue is that you need to specify server side encryption when you create the presigned URL.

GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(
    myBucket, myKey, HttpMethod.PUT)
    .withSSEAlgorithm(SSEAlgorithm.KMS.getAlgorithm());

request.setExpiration(new DateTime().plusMinutes(30).toDate());
request.setContentType("image/jpeg");

URL puturl = s3.generatePresignedUrl(request);

answered Apr 29 '18 at 22:20

John Hanley

74,467
6
95
159

Yes but how do I retrieve the value of `myKey`? When I set up encryption on the bucket in the S3 console, I just chose the default / precreated KMS key. What client code do I need to utilize that? – soapergem Apr 30 '18 at 04:40
mykey is the filename in your code and not the KMS key. S3 buckets are flat and objects (filenames) are called keys. Key – The name that you assign to an object. You use the object key to retrieve or put the object. – John Hanley Apr 30 '18 at 06:30
I included code to interface with KMS SSE. Look at the documentation for withSSEAlgorithm(). https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ServerSideEncryptionByDefault.html – John Hanley Apr 30 '18 at 06:33
Tried that out but now I'm just getting an XML error that says "The request signature we calculated does not match the signature you provided. Check your key and signing method" – soapergem Apr 30 '18 at 13:29
So interestingly enough, when I use the CLI I can generate a presigned URL by first calling `aws configure set default.s3.signature_version s3v4` and then calling `aws s3 presign bucket/filename.jpg` and that URL *works*. However when I use the code above it *does not work*. The only observable difference I see between the two URLs is that the CLI sets the `X-Amz-SignedHeaders` header value to `host` while your code sets it to `content-type;host;x-amz-server-side-encryption` Any thoughts? – soapergem Apr 30 '18 at 19:27
I am going to code this up later today and look at a Wireshark trace. I do most of my work in C++. I will post again after this is complete. When uploading an object those header values are important. The error that you are seeing means that the headers used for signing do not match the headers and / or method used by the object upload. – John Hanley Apr 30 '18 at 20:38
Signature v4 is the default for all AWS SDKs and is used when signing API requests to AWS services but, importantly, it is not necessarily the default signing method used when generating an S3 pre-signed URL. For example, boto3 defaults to v2. Other SDKs, including Java, may do likewise but it's quite hard to find any concrete documentation here. – jarmod Feb 04 '20 at 18:51

How do I utilize AWS Signature v4 when generating a presigned S3 URL?

1 Answers1