2

I'm storing large datasets in s3 and want to create presigned urls to hand out to clients who want to download selected columns from a dataset. The (java) sdk does not seem to offer a pre-packaged way to do this.

Has Amazon made any explicit statement about using s3 select with a presigned url? I couldn't find anything by googling or browsing docs.

Flailing about, I sent a request to a presigned url generated by the sdk with an XML request body (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectSELECTContent.html) for an s3 select, but I get back a SignatureDoesNotMatch Error response. Maybe I need to modify the authentication parameters because I'm changing the content and content-type. Am I on a wild goose chase here or could something like this possibly work?

Is there an easier way? Or is it entirely unsupported?

Eddie Carlson
  • 339
  • 3
  • 6
  • Did you try [Generate a presigned Object URL Using the AWS SDK for Java](https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURLJavaSDK.html), or I misunderstood the question? (just to clarify - I know the pain of working with AWS docs) – the0ffh Apr 30 '19 at 23:10
  • @the0ffh yes, I have created presigned urls using the method in the doc you linked. Using the link, I can download a full dataset I've stored in s3. What I want to know is if I can somehow attach an s3-select expression to it to create a request that will result in the downloading of only the columns specified in the select expression. – Eddie Carlson Apr 30 '19 at 23:13
  • 1
    You can pre-sign a URL for just about anything, so this is *probably* possible... but that's not the big problem. Read further down the document. The response is binary, and uses a custom format to stream the data back to the client. A browser won't handle this properly. – Michael - sqlbot May 01 '19 at 00:42
  • @Michael-sqlbot Fair enough, but being able to stream the data would be a start. I could maybe write a client library to interpret the response from s3. Basically I just want to avoid being the middle-man between clients and s3. – Eddie Carlson May 01 '19 at 18:12
  • this is not pre-signed url but I think you can start from here. Please see Selecting [Content from Objects Using the SDK for Java](https://docs.aws.amazon.com/AmazonS3/latest/dev/SelectObjectContentUsingJava.html) – Rodel Aug 21 '19 at 05:36

0 Answers0