1

I'm wondering what the best way is to upload a video to S3 via a presigned URL. I am primarily considering using a standard HTTP PUT request, placing video/mp4 as the Content-Type, and then attaching the video file as the body.

I'm wondering if there are more efficient approaches to doing this, such as using a third party library or possibly compressing the video before sending it via the PUT request?

Victor M
  • 603
  • 4
  • 22
  • Best is a loaded question, but I'd go for a [multipart upload](https://stackoverflow.com/questions/31799982/aws-sdk-presigned-url-multipart-upload), even if it is harder to do with presigned URLs, since it lets the uploader retry small failures, and run with concurrent uploads of the parts. And compressing video data is generally a pointless activity. – Anon Coward Mar 10 '22 at 04:22
  • Got it, thanks. Do you know if it is possible to utilize this and Cognito at the same time? Only want authenticated users to be able to upload. – Victor M Mar 11 '22 at 20:34

1 Answers1

2

In general, when your object size reaches 100 MB, you should consider using multipart uploads instead of uploading the object in a single operation.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

I had most success using Uppy for this

https://uppy.io/docs/aws-s3-multipart/

You will need to provide some backend endpoints though:

  1. https://uppy.io/docs/aws-s3-multipart/#createMultipartUpload-file
  2. https://uppy.io/docs/aws-s3-multipart/#listParts-file-uploadId-key
  3. https://uppy.io/docs/aws-s3-multipart/#prepareUploadParts-file-partData
  4. https://uppy.io/docs/aws-s3-multipart/#abortMultipartUpload-file-uploadId-key
  5. https://uppy.io/docs/aws-s3-multipart/#completeMultipartUpload-file-uploadId-key-parts

On compression part of you question, S3 does not have any compute. It will not modify your sent bytes, it will just store it. If want to use compression, you need to do that before upload, upload to cloud, unzip there with some compute (Ec2, Lambda etc.) and then put to S3.

Lukas Liesis
  • 24,652
  • 10
  • 111
  • 109
  • Thanks for this. I'm now wondering: how does one incorporate authorization into this? If we set the CORS policy to allow for any PUT/POST requests, ideally I'd like to make it so that only signed in users via Cognito can make these PUT requests, but all users (including those not signed in) can make GET requests. Is there a solution you would recommend? – Victor M Mar 11 '22 at 17:02
  • Also, I see Uppy generates presigned URLs for the multipart upload, but in the actual AWS documentation, it seems they use s3Client to do the uploading. Is one way better than the other? – Victor M Mar 11 '22 at 17:33
  • if you want to authenticate with cognito, consider publishing api gateway deployment with cognito authentication to get the signed URLs needed for upload. If you will do upload itself through api gateway, you will hit payload limit of 6 or 10Mb iirc. Uppy provides it's own backend service iirc, which i never used. And i'm pretty sure they won't support cognito. If you find my posts useful, please consider upvoting them :) – Lukas Liesis Mar 16 '22 at 10:34