Prevent Javascript adding metadata on File Upload to S3

Question

Basically I am willing to upload files directly to S3 via browser i.e without any web server acting as a middle-ware or proxy like this.

So I am generating pre-signed URL using boto3 library like this:

def put_url(self, key):
    url = self.client.generate_presigned_url(
        ClientMethod="put_object",
        Params={
            "Bucket": "visweswaran",
            "Key": key
        }
    )
    return url

and this returns a pre-signed URL which is completely fine. I am using JQuery to make ajax PUT request to the S3 to upload my file.

                let file_data = document.getElementById("file_data").files[0];
                var form = new FormData();
                form.append("", file_data, "test.txt");
                var settings = {
                  "url": url,
                  "method": "PUT",
                  "timeout": 0,
                  "processData": false,
                  "mimeType": "multipart/form-data",
                  "contentType": "text/plain",
                  "beforeSend": function(xhr){xhr.setRequestHeader('Content-Disposition', 'attachment');},
                  "data": form
                };
                $.ajax(settings).done(function (response) {
                  location.reload();
                });

The file gets uploaded to the S3 successfully via browser. But when I open the file I see strange meta data getting added to the top of the file like this,

-----------------------------33057860671031084693134041830 Content-Disposition: form-data; name="name"

test.txt -----------------------------33057860671031084693134041830 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain

I have also tried a more formal solution like Pluploader (https://www.plupload.com/) and I am facing the same problem. I would like somebody to point me in the right direction to fix it. Ant help is much appreciated.

References:

Working Solution

I have tested with a video and you don't need a form. Just send the data directly

let video = document.getElementById("video_file").files[0];
var settings = {
    "url": url,
    "method": "PUT",
    "timeout": 0,
    "processData": false,
    "data": video
};
$.ajax(settings).done(function (response) {
    location.reload();
});

I recommend using POST instead of PUT to upload files from browser. You can use `generate_presigned_post` instead to generate required value of the fields for the frontend form. Here is a great [tutorial](https://blog.webiny.com/upload-files-to-aws-s3-using-pre-signed-post-data-and-a-lambda-function-7a9fb06d56c1) of how to do it step by step. — jellycsc, Jul 16 '20 at 17:36
Could you try sending the PUT request without using formData at all? The `data` attribute would have the reference of `file_data`, and the content-type while the signing and sending (ajax) can be `ContentType: 'binary/octet-stream'`. — Mu-Majid, Jul 18 '20 at 18:42
What happens if you remove `"mimeType": "multipart/form-data",` and `xhr.setRequestHeader('Content-Disposition', 'attachment');` ? — Felix Kling, Jul 18 '20 at 19:01
@FelixKling I have removed the mimeType and I am seeing the same thing -----------------------------261461442633876784451364777288 Content-Disposition: form-data; name=""; filename="test.txt" Content-Type: text/plain — VISWESWARAN NAGASIVAM, Jul 18 '20 at 19:11

Mu-Majid · Accepted Answer · 2020-07-19T10:03:43.560

1

I have tried uploading a txt file with a presigned-put-url using two approaches:

Sending A form data: (this is used with POST urls not PUT)

This actually add the content-disposition header to the final file as mentioned in the question.

Sending raw binary data (recommended way and this how PUT url is used!):

The file was uploaded correctly, and does not include the content-disposition header.

Could you try sending the PUT request without using formData at all?

The ajax's data attribute should have a value of file_data, and the content-type while the signing the S3 URL and sending (ajax) should be ContentType: 'binary/octet-stream'.

If you need to use formData, check out S3's preSignedPost.

edited Jul 19 '20 at 10:03

answered Jul 18 '20 at 18:55

Mu-Majid

851
1
9
16

I quiet don't understand your answer, I have changed the content type but data attribute is where the content of the file goes or am I missing something? – VISWESWARAN NAGASIVAM Jul 18 '20 at 19:19
1

Yes, data attribute is where the content of the file goes. All i was trying to say is you should send the raw binary data of the file in that attribute, and not a formData object. I have edited my answer. – Mu-Majid Jul 19 '20 at 09:57

Prevent Javascript adding metadata on File Upload to S3

1 Answers1