Amazon S3: automatic deletion of certain files with s3cmd

Question

I need a script for s3cmd that sends the deletion command for some files in my Amazon S3 bucket. I know there's this method built-in but it's not useuful. Let's see why.

At the moment I have a script that backups my database every midnight and my websites files every Friday and Tuesday (still at midnight).

The script I'm looking for should delete files in order to have on the bucket only: 1 backup per month (let's say, of the 1st day of every month) of both db and files. And leave all the backups (every day for db and tuesday and friday for files, so the "default" behavior) for the last 7 days.

Example, if I'm on june 17th 2012 the situation should be:

1st January 2012: db and files
1st february 2012: db and files
1st march 2912: db and files
1st april 2012: db and files
1st may 2012: db and files
10th june 2012: db
11th june 2012: db
12th june 2012: db and files
13th june 2012: db
14th june 2012: db
15th june 2012: db and files
16th june 2012: db

Then on july 2nd 2012 the bucket should contain:

1st of jan, feb, march, apr, may, june: both db and files
last 7 days: "default" backups untouched (files backup only on tuesday and friday and db every midnight).

The script would automate what I'm doing manually right now (I already have just 1 backup per past month, so I have to delete backups starting from 7 days ago going back to the last 1st-of-the-month backup).

Sorry for the convoluted question, I hope it's clear enough. :D

score 2 · Answer 1 · answered Apr 11 '13 at 17:49

2

To configure s3cmd in order to be able to run in in your scripts run:

s3cmd --configure

Run this command from the user who has to run the backup script or copy the generated file to that user home directory.

To delete file with s3cmd you use the del command, so, in your script you should call s3cmd like this:

s3cmd del s3://bucketname/file

Of course, you can also use wildcards:

s3cmd del s3://bucketname/file*

You can name you backup files with the date:

db.backup.20130411.tgz

and use the date in order to delete previous backups.

Btw, in the s3ql project there is an expire_backups script that might be doing already what you want to do.

answered Apr 11 '13 at 17:49

adosaiguas

1,331
9
13

I know how to call s3cmd and that I delete files with del, but I'm looking for the exact script to delete those files I explained :-) – MultiformeIngegno Apr 11 '13 at 23:25
... and have the situation I explained mantained periodically. :) – MultiformeIngegno Apr 11 '13 at 23:51
1

Well, this is a questions and answers site, and not a place where to request people program for you. There are other sites where you can do that. – adosaiguas Apr 11 '13 at 23:54
Yours isn't an answer as what you said was already covered in the question – MultiformeIngegno Apr 12 '13 at 01:12
"A detailed canonical answer is required to address all the concerns." – MultiformeIngegno Apr 12 '13 at 01:24

Amazon S3: automatic deletion of certain files with s3cmd

1 Answers1