How to download files in an S3 bucket

Question

How to copy all files that start with a prefix. I have a lot of files with the name 2021 - ** - **. You need to download them for a certain period. This commands did not help

aws s3 sync s3://my-bucket/2021-04-12-* .
aws s3 cp s3://my-bucket/2021-04-12-* .

possible duplicate https://stackoverflow.com/a/44586711/13126651 — Jatin Mehrotra, Apr 21 '21 at 14:30
Does this answer your question? [copy data from s3 to local with prefix](https://stackoverflow.com/questions/44583592/copy-data-from-s3-to-local-with-prefix) — Josh, Apr 21 '21 at 17:55

score 0 · Answer 1 · answered Apr 21 '21 at 15:41

Use python3 script

import boto3
import os

s3 = boto3.resource('s3')
my_bucket = s3.Bucket('my-bucket')

for my_bucket_object in my_bucket.objects.all():
    # Assume your bucket structure as s3://my-bucket/<files> 
    if my_bucket_object.key.startswith('2021-04-12-'):
        os.system(f'aws s3 cp s3://my-bucket/{my_bucket_object.key} .')

How to download files in an S3 bucket

1 Answers1