3

I'm writing a service that takes screenshots of a lot of URLs and saves them in a public S3 bucket.
Due to storage costs, I'd like to periodically purge the aforementioned bucket and delete every screenshot that hasn't been accessed in the last X days.
By "accessed" I mean downloaded or acquired via a GET request.

I checked out the documentation and found a lot of ways to define an expiration policy for an S3 object, but couldn't find a way to "mark" a file as read once it's been accessed externally.

Is there a way to define the periodic purge without code (only AWS rules/services)? Does the API even allow that or do I need to start implementing external workarounds?

KidCrippler
  • 1,633
  • 2
  • 19
  • 34
  • I'm intersted in this too, did you find a solution delete files that haven't been accessed in some period? – SimonW Dec 15 '19 at 11:24

2 Answers2

2

You can use Amazon S3 Storage Class Analysis:

By using Amazon S3 analytics storage class analysis you can analyze storage access patterns to help you decide when to transition the right data to the right storage class. This new Amazon S3 analytics feature observes data access patterns to help you determine when to transition less frequently accessed STANDARD storage to the STANDARD_IA (IA, for infrequent access) storage class.

After storage class analysis observes the infrequent access patterns of a filtered set of data over a period of time, you can use the analysis results to help you improve your lifecycle policies.

Even if you don't use it to change Storage Class, you can use it to discover which objects are not accessed frequently.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • It looks like something that generates reports, as opposed to an API that I can call on demand. Am I missing something? – KidCrippler Nov 05 '18 at 23:25
  • 1
    The page says: "You can choose to have storage class analysis export analysis reports to a comma-separated values (CSV) flat file. Reports are updated daily and are based on the object age group filters you configure." Thus, you would need to parse the file to obtain the information. – John Rotenstein Nov 06 '18 at 00:46
0

There is no such service provided by AWS.. You will have to write your own solution.

Deepak Singhal
  • 10,568
  • 11
  • 59
  • 98