Can I use Glacier to snapshot S3 like a point-in-time backup?

Question

I have a series of directories on numerous computers that I backup to an S3 bucket nightly using dgsync. I'd like a solution that will permit me to archive occasionally:

once a week for the past four weeks
once a month for the past six months

The price point of Glacier is attractive, but I'm curious to know if I can do that with a Lifecycle Policy. Or, if there is another utility I have to use for the migration.

Beware that archiving small objects to Glacier may be more expensive than leaving them in S3. Here's an article I wrote with more info: http://alestic.com/2012/12/s3-glacier-costs "Objects 13 KB or smaller are going to cost more to transition to Glacier rather than leaving in S3." Larger objects can still take a long time to break even in Glacier. — Eric Hammond, Jun 19 '13 at 01:19
But the *primary* goal isn't as much to save money. It's to archive numerious point-in-time-ish backups. — Evan Carroll, Jun 19 '13 at 03:03
Then no, lifecycle policies will not help you active your goals. You will need to implement the archiving yourself. — Eric Hammond, Jun 19 '13 at 11:02

score 2 · Answer 1 · answered Jun 19 '13 at 11:06

No, S3 lifecycle policies cannot be used to create multiple point in time snapshots of S3.

Lifecycle policies can be used to move S3 objects to Glacier automatically , but the originals are no longer accessible in S3 storage without first restoring from Glacier.

score 0 · Answer 2 · answered Jun 19 '13 at 01:04

0

Yes, you can do this with S3 bucket lifecycle policies. The documentation spells it all out in detail. In short you need to specify:

A object key prefix that identifies one or more objects to which the rule applies.

An action or actions that you want Amazon S3 to perform on the specified objects.

A date or a time period, specified in days since object creation, when you want Amazon S3 to perform the specified action.

In your case, in the action, you'd specify the storage class as "glacier".

I should mention that this will surely not be a true point-in-time snapshot. If any of the target objects are modified while they're being archived (unlikely), the state will not be consistent.

answered Jun 19 '13 at 01:04

EEAA

109,363
18
175
245

How is that, I thought S3 was atomic (though eventual) to begin with? – Evan Carroll Jun 19 '13 at 01:09
Individual objects, yes. Not sets of objects. – EEAA Jun 19 '13 at 01:15
Note that with this approach, the objects are no longer stored in S3 after moving to Glacier. If you wish to access them after archival, you'll need to pay the charges and wait the time to bring them from Glacier back to S3. – Eric Hammond Jun 19 '13 at 01:15
Does Glacier report a *copy-to* Glacier, without a *remove* from S3? – Evan Carroll Jun 19 '13 at 14:50
@EvanCarroll - yes of course, but not via lifecycles. You'd need to implement the copy on your own. – EEAA Jun 19 '13 at 17:01

Can I use Glacier to snapshot S3 like a point-in-time backup?

2 Answers2