Storing a lot of images S3 vs DynamoDB

Question

I am working on project that needs to store a lot of images per second. And I'm a little bit confused if I should store my images in S3 or in DynamoDB. My confusion is because it seems that S3 do not support batch requests and I have to be able to get all images in a batch,if necessary; On the other hand I'm not sure if it is a good idea store all these images in a table in dynamoDB. What would be the best approach?

Answer 1

Short answer: Use S3.

Long answer: DynamoDB limits individual records to 64 KB, so you probably can't use DynamoDB unless your images are tiny anyway.

You mention wanting to read your images in a batch request, but DynamoDB batch requests return up to 100 records or 1 MB -- whichever is less -- so it's not like you can retrieve all your images at once anyway. You could dump everything from DynamoDB by reading a page at a time, which means lots of requests one after another. Not fast.

DynamoDB is lower latency than S3, and it supports configurable throughput. However, S3 supports as much concurrency as you want out of the box. Want to upload 1000 images at once? Go for it! It doesn't matter if each object takes 400 ms to write since you can transmit as many as you want concurrently. Similarly, you can dump everything from S3 by getting a list of everything in the bucket and then retrieving every object in parallel.

Other reasons to use S3:

• HTTP compatibility, so you can point other people or applications straight to the bucket
• Hugely lower storage costs
• Pay per request, not for provisioned throughput (640 KB/s of write capacity in DynamoDB costs $460/month, versus a flat $0.01 per 1000 uploads in S3)

The only situation I can picture where DynamoDB might make sense for image storage is if your images are small, frequently changing, and you're very sensitive to read latency. Other than that, use S3.

Answer 2

Store the images in S3. The DynamoDB limitations have been described by willglynn. However you may want to store the urls to the images and other metadata in your DynamoDB.

A bit of warning about DynamoDB, it can be very expensive. Going up from free-tier may easily cost a few hundred dollars for a dozen more throughput units. If you used it for storing large items (close to 64KB such as images) which need even larger capacity, it's very likely you would end up paying thousand of dollars. For storing metadata with AWS, you may want to check SimpleDB.