1

I have tried several packages (goofys, s3fs, fuse) and have not been able to find a solution to my problem.

I have an S3 Bucket with several thousand files currently in it and will continue to grow. Most applications to map S3 to EC2 work for smaller foot prints than what I am dealing with. The delivery of the files is not of my choice.

All of the files are less than 100KB. The greatest issue that I have found thus far is the large number of files. I need a means for a PHP script to poll the list of files looking for common patterns that are known to build a stored list for future reference. i.e. IMG-20160704_01.png In this case I am looking for all files containing 20160704.

Anyone with current working solution(s) for this?

EC2 is Centos 6.5

Thank you.

Andrew Gaul
  • 262
  • 1
  • 7
Steven K7FAQ
  • 277
  • 2
  • 3
  • 13

1 Answers1

1

It can easily be done via any of AWS SDK's. You need to do the following:

  1. Retreive a list of all objects located in S3 bucket.
  2. Filter the list according to your criteria using regular expressions.
  3. Dowload files using the filnal list of objects

If you are using PHP, the following docs could be useful for you: http://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3-2006-03-01.html

Note: I can do it on python for you. Drop me a note if you are interested (my contact email is in the profile description)

Vladimir Mukhin
  • 210
  • 1
  • 3
  • Thanks, I will keep this in mind. I have built several similar type apps since this 3 year old posting. I have done it using both PHP and Python. – Steven K7FAQ Mar 26 '19 at 13:29