0


I'm currently building a backend application that will need to store and serve a lot of images and videos (> 1 Mio. each). The files will obviously get uploaded and sometimes deleted but not updated.

I did some reading on the issue of whether to store the files in my database or in the file system and decided to use the file system for file storage (and keep metadata in the DB).

How I upload files:
I'm uploading files synchronously via a REST endpoint on the file server.
How I delete files:
I delete files asynchronously by marking them as obsolete in the DB metadata and then letting a cron job in the file server do the actual deleting.

Now at the moment I'm trying to find out how I would backup my database and file server in a synchronized fashion. I'm not much of a devOps guy and didn't find anything useful during my Google researches (probably because I lack the right terminology for a good question).

My current plan:

currently my naiv idea is to always first backup the database metadata and once that finishes to backup the files. I suspend my garbage collector for the time of the file backup. If I restore my DB, there might be orphaned files on the file system but that's ok (I guess ...)

Are there logic holes in my current strategy? What better options are there? At the moment I'm not using any cloud storage solution so please don't suggest stuff that e.g. Amazon offers for services like S3

Thanks!

Wecherowski
  • 818
  • 11
  • 24

1 Answers1

1

The simple thing you can do is to set an 'expired' date instead of 'obsoletFlag' and consider a backup maximum period less than the expiration period.

Adapt your delete job to process only 'expired' items.

You will have to store for a longer period but any not expired backup could be restore accordingly.

Regards.

Nicolas COSME
  • 99
  • 3