uploading file to specific folder in S3 using boto3

Question

Everything with my code works. The only pitfall I am currently facing is that I cannot specify the folder within the S3 bucket that I would like to place my file in. Here is what I have:

s3.meta.client.upload_file('/tmp/'+filename, '<bucket-name>', filename)

I have tried both:

s3.meta.client.upload_file('/tmp/'+filename, '<bucket-name>/folder/', filename)

and:

s3.meta.client.upload_file('/tmp/'+filename, '<bucket-name>', '/folder/'+filename)

if anyone has any tips on how to direct this to a specific folder (if this is possible) please let me know!

S3 is a object store, it doesn't deal with "folder name". The whole "path" must be use as part of the key name. — mootmoot, Sep 01 '16 at 13:22

score 70 · Accepted Answer · answered Sep 01 '16 at 13:15

70

You do not need to pass the Key value as an absolute path. The following should work:

upload_file('/tmp/' + filename, '<bucket-name>', 'folder/{}'.format(filename))

answered Sep 01 '16 at 13:15

hjpotter92

78,589
36
144
183

score 10 · Answer 2 · answered Sep 01 '16 at 14:00

10

I figured out my problem. I had the right idea with the /folder/ option in the key parameter area, however, I did not need the first / Thank you all! This is essentially the same idea as hjpotter92's suggestion above.

answered Sep 01 '16 at 14:00

Alex

493
1
4
9

score 8 · Answer 3 · edited May 08 '19 at 23:07

8

Use this for python 3.x

s3.upload_file(file_path,bucket_name, '%s/%s' % (bucket_folder,dest_file_name))

edited May 08 '19 at 23:07

somesingsomsing

3,182
4
29
46

answered Oct 12 '18 at 13:04

Andrew Figaroa

81
1
3

this worked for me. I used another s3 function (`s3.upload_fileobj`) ad it worked as well without any issue. Many thanks! – alonso_50 Jul 25 '23 at 10:21

score 2 · Answer 4 · answered Nov 24 '20 at 21:21

2

You can use put_object in the place of upload_file:

file = open(r"/tmp/" + filename)
response = s3.meta.client.Bucket('<bucket-name>').put_object(Key='folder/{}'.format(filename), Body=file)

answered Nov 24 '20 at 21:21

OM Bharatiya

1,840
14
23

this worked for me although my syntax looks a bit different : s3.Bucket("bucketname").put_object(Key='public/' + key, Body=data) – elesh.j Aug 25 '22 at 16:05

score 1 · Answer 5 · answered Nov 04 '17 at 14:16

Here is the method that will take care of nested directory structure, and will be able to upload a full directory using boto

def upload_directory():
    for root, dirs, files in os.walk(settings.LOCAL_SYNC_LOCATION):
        nested_dir = root.replace(settings.LOCAL_SYNC_LOCATION, '')
        if nested_dir:
            nested_dir = nested_dir.replace('/','',1) + '/'

        for file in files:
            complete_file_path = os.path.join(root, file)
            file = nested_dir + file if nested_dir else file
            print "[S3_UPLOAD] Going to upload {complete_file_path} to s3 bucket {s3_bucket} as {file}"\
                .format(complete_file_path=complete_file_path, s3_bucket=settings.S3_BUCKET, file=file)
            s3_client.upload_file(complete_file_path, settings.S3_BUCKET, file)

score 0 · Answer 6 · answered Feb 18 '21 at 12:40

0

Try it:

s3.meta.client.upload_file(Filename=filename_and_full_path, Bucket=my_bucket, Key=prefix_key_plus_filename_only)

answered Feb 18 '21 at 12:40

Neto Ribeiro

1

uploading file to specific folder in S3 using boto3

6 Answers6

Use this for python 3.x