0

My use cases:

  1. I would like to store multiple versions of text files in Cloud Storage and save the createdAt timestamp at which each version was created.
  2. I'd like to be able to retrieve a list of all versions and createdAt times without opening and reading the files.
  3. I'd also like to create nightly backups of the bucket with all the versions intact, and for each file to keep a record of its original createdAt time.

My solutions:

  1. I have a Cloud Storage bucket with versioning enabled. Every time a I save a file test, I get a new file test#generation_number.
  2. I can list all versions and fetch an older version with test#generation_number
  3. I can back up all versions of test in the bucket, using gsutil cp -A gs://my-original-bucket/test gs://my-backup-bucket.

The issue with point #3. The #generation_number of each version that is backed up changes to the time at which the version of each backup file was created, not the time of the original file creation. I understand this is working as intended, and the order of the versions is still intact.

But where to stash these original createdAt values? I tried to store them in metadata, and I found metadata seems not to be versioned, but rather global for the file object as a whole.

What is the best way to achieve my use case? Is there any way to do so directly with Google Cloud Storage? Or should I maintain a separate database with this information, instead?

wafflefry
  • 1
  • 1
    Howdy and welcome to Stackoverflow. Got to thinking about your puzzle and a question comes to mind. Given that Google gives us 11 9's of durability of the data and every time you make a new save, the old version is also stored ... what is the higher level thinking of you creating a "backup" yourself? What does a manually created backup give you that you don't get without even taking a backup in the first place? – Kolban May 28 '19 at 21:48
  • @Kolban Hmm, indeed my bucket is multi-regional, so it's already geo-redundant. I suppose my thinking is just the paranoid programmer in me saying "you should really back this up somewhere else, just in case." You make a good point: maybe I'm paying the platform to offload that "just in case", and the solution for my "problem" is not obvious because the problem doesn't really exist! I'll think on it a bit more, but that would indeed be a happy conclusion. – wafflefry May 28 '19 at 22:48

0 Answers0