0

I'm creating a filesystem and I think I'll be storing files in a DB (http://sietch.net/ViewNewsItem.aspx?NewsItemID=124 and http://blog.druva.com/2009/01/25/file-systems-vs-databases/ seem to indicate it's a good idea).

Since it's a filesystem, I'll need A LOT of I/O and REALLY fast. If I'm hosting on EC2, will Amazon SimpleDB be a decent solution for this?

Shamoon
  • 41,293
  • 91
  • 306
  • 570
  • 1
    Why do you need to create your own filesystem? There are already some great distributed filesystems (GridFS, HDFS). If you are using EC2, then you could also consider using S3. – mikerobi Jun 01 '11 at 20:29
  • CONSTANT versioning of files. – Shamoon Jun 01 '11 at 21:14

2 Answers2

1

SimpleDB has a maximum record size of a 1,000 BYTES so it is VERY poorly suited to storing files/blobs (unless they are tiny).

It is fairly common for people to use SimpleDB to index files and then to store the files in S3 which is much better suited for storing large objects.

Scrappydog
  • 2,864
  • 1
  • 21
  • 23
  • Will this work when there are hundreds of revisions per file times many files? – Shamoon Jun 03 '11 at 15:48
  • 1
    Not sure what issues you are concerned about, but hundreds of revisions doesn't raise any red flags in my mind... FYI: S3 will let you overwrite a file OR maintain a version history of changes. – Scrappydog Jun 05 '11 at 17:41
0

SimpleDB (and recent DynamoDB) aren't suited to store files. Why don't you just make the versioning control on one of them, indexing files stored on S3?

You don't need to override the files on S3, as you can name them whatever you want and get the original name (and other info) from the database. You can even, for text files, for example, have a preview or the begining of the file on the DB or, for images, have a thumbnail on S3 and also get this info from the DB, so when users list the files they get only the thumbnail and, if they want, they download the full-size file.

Take a look at http://aws.amazon.com/en/dynamodb/faqs/#When_should_I_use_Amazon_DynamoDB_vs_Amazon_S3 and http://aws.amazon.com/en/running_databases/

tvdias
  • 821
  • 2
  • 10
  • 25