How to access a file in a folder in a bucket in boto using Python?

Question

All I could find was this, which lists all the files. The listing succeeds.

I want to just access a specific file by its link. The file is in a folder.

For example, I want to download from the following URL:

s3://my_bucket/my_folder/my_next_folder/my_file.csv

Without traversing the entire file tree.

score 0 · Answer 1 · answered Aug 08 '20 at 10:19

Have you try this:

import boto3
import botocore

BUCKET_NAME = 'my-bucket' # replace with your bucket name
KEY = 'my_image_in_s3.jpg' # replace with your object key

s3 = boto3.resource('s3')

try:
    s3.Bucket(BUCKET_NAME).download_file(KEY, 'my_local_image.jpg')
except botocore.exceptions.ClientError as e:
    if e.response['Error']['Code'] == "404":
        print("The object does not exist.")
    else:
        raise

score 0 · Accepted Answer · answered Aug 09 '20 at 06:29

from boto.s3.key import Key
from boto.s3.connection import S3Connection
conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
s3_url_split = s3_url.split('/')
bucket_name = s3_url_split[2]
dir_name = '/'.join(s3_url_split[3:-1])

bucket = conn.get_bucket(bucket_name)
file_name = s3_url_split[-1]
k = Key(bucket)
k.key = dir_name + "/" + file_name
k.get_contents_to_filename(dst_path)

Thing I was missing was the key already contains the folder path.

How to access a file in a folder in a bucket in boto using Python?

2 Answers2